Grammatical evolution hyper-heuristic for combinatorial optimization problems

Designing generic problem solvers that perform well across a diverse set of problems is a challenging task. In this work, we propose a hyper-heuristic framework to automatically generate an effective and generic solution method by utilizing grammatical evolution. In the proposed framework, grammatical evolution is used as an online solver builder, which takes several heuristic components (e.g., different acceptance criteria and different neighborhood structures) as inputs and evolves templates of perturbation heuristics. The evolved templates are improvement heuristics, which represent a complete search method to solve the problem at hand. To test the generality and the performance of the proposed method, we consider two well-known combinatorial optimization problems: exam timetabling (Carter and ITC 2007 instances) and the capacitated vehicle routing problem (Christofides and Golden instances). We demonstrate that the proposed method is competitive, if not superior, when compared to state-of-the-art hyper-heuristics, as well as bespoke methods for these different problem domains. In order to further improve the performance of the proposed framework we utilize an adaptive memory mechanism, which contains a collection of both high quality and diverse solutions and is updated during the problem solving process. Experimental results show that the grammatical evolution hyper-heuristic, with an adaptive memory, performs better than the grammatical evolution hyper-heuristic without a memory. The improved framework also outperforms some bespoke methodologies, which have reported best known results for some instances in both problem domains

The General Combinatorial Optimization Problem: Towards Automated Algorithm Design

This paper defines a new combinatorial optimisation problem, namely General Combinatorial Optimisation Problem (GCOP), whose decision variables are a set of parametric algorithmic components, i.e. algorithm design decisions. The solutions of GCOP, i.e. compositions of algorithmic components, thus represent different generic search algorithms. The objective of GCOP is to find the optimal algorithmic compositions for solving the given optimisation problems. Solving the GCOP is thus equivalent to automatically designing the best algorithms for optimisation problems. Despite recent advances, the evolutionary computation and optimisation research communities are yet to embrace formal standards that underpin automated algorithm design. In this position paper, we establish GCOP as a new standard to define different search algorithms within one unified model. We demonstrate the new GCOP model to standardise various search algorithms as well as selection hyper-heuristics. A taxonomy is defined to distinguish several widely used terminologies in automated algorithm design, namely automated algorithm composition, configuration and selection. We would like to encourage a new line of exciting research directions addressing several challenging research issues including algorithm generality, algorithm reusability, and automated algorithm design

An investigation of the ant-based hyper-heuristic for capacitated vehicle routing problem and traveling salesman problem

A brief observation on recent research of routing problems shows that most of the methods used to tackle the problems are using heuristics and metaheuristics; and they often use problem specific knowledge to build or improve solutions. In the last few years, research on hyper-heuristic has been investigated which aims to raise the generality of optimisation systems. This thesis is concerned with the investigation of ant-based hyper-heuristic. Ant algorithms have been applied to vehicle routing problems and have produced competitive results. Therefore, it is assumed that there is a reasonable possibility that ant-based hyperheuristic could perform well for the problem. The thesis first surveys the literature for some common solution methodologies for optimisation problems and explores in some detail the ant algorithms and ant algorithm hyperheuristic methods. Furthermore, the literature specifically concerns with routing problems; the capacitated routing problem (CVRP) and the travelling salesman problem (TSP). The thesis studies the ant system algorithm and further proposes the ant algorithm hyper-heuristic, which introduces a new pheromone update rule in order to improve its performance. The proposed approach, called the ant-based hyper-heuristic is tested to two routing problems; the CVRP and TSP. Although it does not produce any best known results, the experimental results have shown that it is competitive with other methods. Most importantly, it demonstrates how simple and easy to implement low level heuristics, with no extensive parameter tuning. Further analysis shows that the approach possesses learning mechanism when compared to random hyper-heuristic. The approach investigates the number of low level heuristics appropriate and found out that the more low level heuristics used, the better solution is generated. In addition an ACO hyper-heuristic which has two categories of pheromone updates is developed. However, ant-based hyper-heuristic performs better and this is inconsistent with the performance of ACO algorithm in the literature. In TSP, we utilise two different categories of low level heuristics, the TSP heuristics and the CVRP heuristics that were previously used for the CVRP. From the observation, it can be seen that by using any heuristics for the same class of problems, ant-based hyper-heuristic is seen to be able to produce competitive results. This has demonstrated that the ant-based hyper-heuristic is a reusable method. One major advantage of this work is the usage of the same parameter for all problem instances with simple moves and swap procedures. It is hoped that in the future, results obtained will be better than current results by using better intelligent low level heuristics

On Neighborhood Tree Search

We consider the neighborhood tree induced by alternating the use of different neighborhood structures within a local search descent. We investigate the issue of designing a search strategy operating at the neighborhood tree level by exploring different paths of the tree in a heuristic way. We show that allowing the search to 'backtrack' to a previously visited solution and resuming the iterative variable neighborhood descent by 'pruning' the already explored neighborhood branches leads to the design of effective and efficient search heuristics. We describe this idea by discussing its basic design components within a generic algorithmic scheme and we propose some simple and intuitive strategies to guide the search when traversing the neighborhood tree. We conduct a thorough experimental analysis of this approach by considering two different problem domains, namely, the Total Weighted Tardiness Problem (SMTWTP), and the more sophisticated Location Routing Problem (LRP). We show that independently of the considered domain, the approach is highly competitive. In particular, we show that using different branching and backtracking strategies when exploring the neighborhood tree allows us to achieve different trade-offs in terms of solution quality and computing cost.Comment: Genetic and Evolutionary Computation Conference (GECCO'12) (2012

Reinforcement Learning-assisted Evolutionary Algorithm: A Survey and Research Opportunities

Evolutionary algorithms (EA), a class of stochastic search methods based on the principles of natural evolution, have received widespread acclaim for their exceptional performance in various real-world optimization problems. While researchers worldwide have proposed a wide variety of EAs, certain limitations remain, such as slow convergence speed and poor generalization capabilities. Consequently, numerous scholars actively explore improvements to algorithmic structures, operators, search patterns, etc., to enhance their optimization performance. Reinforcement learning (RL) integrated as a component in the EA framework has demonstrated superior performance in recent years. This paper presents a comprehensive survey on integrating reinforcement learning into the evolutionary algorithm, referred to as reinforcement learning-assisted evolutionary algorithm (RL-EA). We begin with the conceptual outlines of reinforcement learning and the evolutionary algorithm. We then provide a taxonomy of RL-EA. Subsequently, we discuss the RL-EA integration method, the RL-assisted strategy adopted by RL-EA, and its applications according to the existing literature. The RL-assisted procedure is divided according to the implemented functions including solution generation, learnable objective function, algorithm/operator/sub-population selection, parameter adaptation, and other strategies. Finally, we analyze potential directions for future research. This survey serves as a rich resource for researchers interested in RL-EA as it overviews the current state-of-the-art and highlights the associated challenges. By leveraging this survey, readers can swiftly gain insights into RL-EA to develop efficient algorithms, thereby fostering further advancements in this emerging field.Comment: 26 pages, 16 figure

Arc routing problems: A review of the past, present, and future

[EN] Arc routing problems (ARPs) are defined and introduced. Following a brief history of developments in this area of research, different types of ARPs are described that are currently relevant for study. In addition, particular features of ARPs that are important from a theoretical or practical point of view are discussed. A section on applications describes some of the changes that have occurred from early applications of ARP models to the present day and points the way to emerging topics for study. A final section provides information on libraries and instance repositories for ARPs. The review concludes with some perspectives on future research developments and opportunities for emerging applicationsThis research was supported by the Ministerio de Economia y Competitividad and Fondo Europeo de Desarrollo Regional, Grant/Award Number: PGC2018-099428-B-I00. The Research Council of Norway, Grant/Award Numbers: 246825/O70 (DynamITe), 263031/O70 (AXIOM).Corberán, Á.; Eglese, R.; Hasle, G.; Plana, I.; Sanchís Llopis, JM. (2021). Arc routing problems: A review of the past, present, and future. Networks. 77(1):88-115. https://doi.org/10.1002/net.21965S8811577

EMVLight: a Multi-agent Reinforcement Learning Framework for an Emergency Vehicle Decentralized Routing and Traffic Signal Control System

Emergency vehicles (EMVs) play a crucial role in responding to time-critical calls such as medical emergencies and fire outbreaks in urban areas. Existing methods for EMV dispatch typically optimize routes based on historical traffic-flow data and design traffic signal pre-emption accordingly; however, we still lack a systematic methodology to address the coupling between EMV routing and traffic signal control. In this paper, we propose EMVLight, a decentralized reinforcement learning (RL) framework for joint dynamic EMV routing and traffic signal pre-emption. We adopt the multi-agent advantage actor-critic method with policy sharing and spatial discounted factor. This framework addresses the coupling between EMV navigation and traffic signal control via an innovative design of multi-class RL agents and a novel pressure-based reward function. The proposed methodology enables EMVLight to learn network-level cooperative traffic signal phasing strategies that not only reduce EMV travel time but also shortens the travel time of non-EMVs. Simulation-based experiments indicate that EMVLight enables up to a $42.6\%$ reduction in EMV travel time as well as an $23.5\%$ shorter average travel time compared with existing approaches.Comment: 19 figures, 10 tables. Manuscript extended on previous work arXiv:2109.05429, arXiv:2111.0027

Applied (Meta)-Heuristic in Intelligent Systems

Engineering and business problems are becoming increasingly difficult to solve due to the new economics triggered by big data, artificial intelligence, and the internet of things. Exact algorithms and heuristics are insufficient for solving such large and unstructured problems; instead, metaheuristic algorithms have emerged as the prevailing methods. A generic metaheuristic framework guides the course of search trajectories beyond local optimality, thus overcoming the limitations of traditional computation methods. The application of modern metaheuristics ranges from unmanned aerial and ground surface vehicles, unmanned factories, resource-constrained production, and humanoids to green logistics, renewable energy, circular economy, agricultural technology, environmental protection, finance technology, and the entertainment industry. This Special Issue presents high-quality papers proposing modern metaheuristics in intelligent systems

An investigation of the ant-based hyper-heuristic for capacitated vehicle routing problem and traveling salesman problem

A brief observation on recent research of routing problems shows that most of the methods used to tackle the problems are using heuristics and metaheuristics; and they often use problem specific knowledge to build or improve solutions. In the last few years, research on hyper-heuristic has been investigated which aims to raise the generality of optimisation systems. This thesis is concerned with the investigation of ant-based hyper-heuristic. Ant algorithms have been applied to vehicle routing problems and have produced competitive results. Therefore, it is assumed that there is a reasonable possibility that ant-based hyperheuristic could perform well for the problem. The thesis first surveys the literature for some common solution methodologies for optimisation problems and explores in some detail the ant algorithms and ant algorithm hyperheuristic methods. Furthermore, the literature specifically concerns with routing problems; the capacitated routing problem (CVRP) and the travelling salesman problem (TSP). The thesis studies the ant system algorithm and further proposes the ant algorithm hyper-heuristic, which introduces a new pheromone update rule in order to improve its performance. The proposed approach, called the ant-based hyper-heuristic is tested to two routing problems; the CVRP and TSP. Although it does not produce any best known results, the experimental results have shown that it is competitive with other methods. Most importantly, it demonstrates how simple and easy to implement low level heuristics, with no extensive parameter tuning. Further analysis shows that the approach possesses learning mechanism when compared to random hyper-heuristic. The approach investigates the number of low level heuristics appropriate and found out that the more low level heuristics used, the better solution is generated. In addition an ACO hyper-heuristic which has two categories of pheromone updates is developed. However, ant-based hyper-heuristic performs better and this is inconsistent with the performance of ACO algorithm in the literature. In TSP, we utilise two different categories of low level heuristics, the TSP heuristics and the CVRP heuristics that were previously used for the CVRP. From the observation, it can be seen that by using any heuristics for the same class of problems, ant-based hyper-heuristic is seen to be able to produce competitive results. This has demonstrated that the ant-based hyper-heuristic is a reusable method. One major advantage of this work is the usage of the same parameter for all problem instances with simple moves and swap procedures. It is hoped that in the future, results obtained will be better than current results by using better intelligent low level heuristics