A multi-objective and evolutionary hyper-heuristic applied to the Integration and Test Order Problem

The field of Search-Based Software Engineering (SBSE) has widely utilized Multi-Objective Evolutionary Algorithms (MOEAs) to solve complex software engineering problems. However, the use of such algorithms can be a hard task for the software engineer, mainly due to the significant range of parameter and algorithm choices. To help in this task, the use of Hyper-heuristics is recommended. Hyper-heuristics can select or generate low-level heuristics while optimization algorithms are executed, and thus can be generically applied. Despite their benefits, we find only a few works using hyper-heuristics in the SBSE field. Considering this fact, we describe HITO, a Hyper-heuristic for the Integration and Test Order Problem, to adaptively select search operators while MOEAs are executed using one of the selection methods: Choice Function and Multi-Armed Bandit. The experimental results show that HITO can outperform the traditional MOEAs NSGA-II and MOEA/DD. HITO is also a generic algorithm, since the user does not need to select crossover and mutation operators, nor adjust their parameters

Automated design of local search algorithms for vehicle routing problems with time windows

Designing effective search algorithms for solving combinatorial optimisation problems presents a challenge for researchers due to the time-consuming experiments and experience required in decision-making. Automated algorithm design removes the heavy reliance on human experts and allows the exploration of new algorithm designs. This thesis systematically investigates machine learning for the automated design of new and generic local search algorithms, taking the vehicle routing problem with time windows as the testbed. The research starts by building AutoGCOP, a new general framework for the automated design of local search algorithms to optimise the composition of basic algorithmic components. Within the consistent AutoGCOP framework, the basic algorithmic components show satisfying performance for solving the VRPTW. Based on AutoGCOP, the thesis investigates the use of machine learning for automated algorithm composition by modelling the algorithm design task as different machine learning tasks, thus investigating different perspectives of learning in automated algorithm design. Based on AutoGCOP, the thesis first investigates online learning in automated algorithm design. Two learning models based on reinforcement learning and Markov chain are investigated to learn and enhance the compositions of algorithmic components towards automated algorithm design. The Markov chain model presents a superior performance in learning the compositions of algorithmic components during the search, demonstrating its effectiveness in designing new algorithms automatically. The thesis then investigates offline learning to learn the hidden knowledge of effective algorithmic compositions within AutoGCOP for automated algorithm design. The forecast of algorithmic components in the automated composition is defined as a sequence classification task. This new machine learning task is then solved by a Long Short-term Memory (LSTM) neural network which outperforms various conventional classifiers. Further analysis reveals that a Transformer network surpasses LSTM at learning from longer algorithmic compositions. The systematical analysis of algorithmic compositions reveals some key features for improving the prediction. To discover valuable knowledge in algorithm designs, the thesis applies sequential rule mining to effective algorithmic compositions collected based on AutoGCOP. Sequential rules of composing basic components are extracted and further analysed, presenting a superior performance of automatically composed local search algorithms for solving VRPTW. The extracted sequential rules also suggest the importance of considering the impact of algorithmic components on optimisation performance during automated composition, which provides new insights into algorithm design. The thesis gains valuable insights from various learning perspectives, enhancing the understanding towards automated algorithm design. Some directions for future work are present

Automated design of local search algorithms for vehicle routing problems with time windows

Designing effective search algorithms for solving combinatorial optimisation problems presents a challenge for researchers due to the time-consuming experiments and experience required in decision-making. Automated algorithm design removes the heavy reliance on human experts and allows the exploration of new algorithm designs. This thesis systematically investigates machine learning for the automated design of new and generic local search algorithms, taking the vehicle routing problem with time windows as the testbed. The research starts by building AutoGCOP, a new general framework for the automated design of local search algorithms to optimise the composition of basic algorithmic components. Within the consistent AutoGCOP framework, the basic algorithmic components show satisfying performance for solving the VRPTW. Based on AutoGCOP, the thesis investigates the use of machine learning for automated algorithm composition by modelling the algorithm design task as different machine learning tasks, thus investigating different perspectives of learning in automated algorithm design. Based on AutoGCOP, the thesis first investigates online learning in automated algorithm design. Two learning models based on reinforcement learning and Markov chain are investigated to learn and enhance the compositions of algorithmic components towards automated algorithm design. The Markov chain model presents a superior performance in learning the compositions of algorithmic components during the search, demonstrating its effectiveness in designing new algorithms automatically. The thesis then investigates offline learning to learn the hidden knowledge of effective algorithmic compositions within AutoGCOP for automated algorithm design. The forecast of algorithmic components in the automated composition is defined as a sequence classification task. This new machine learning task is then solved by a Long Short-term Memory (LSTM) neural network which outperforms various conventional classifiers. Further analysis reveals that a Transformer network surpasses LSTM at learning from longer algorithmic compositions. The systematical analysis of algorithmic compositions reveals some key features for improving the prediction. To discover valuable knowledge in algorithm designs, the thesis applies sequential rule mining to effective algorithmic compositions collected based on AutoGCOP. Sequential rules of composing basic components are extracted and further analysed, presenting a superior performance of automatically composed local search algorithms for solving VRPTW. The extracted sequential rules also suggest the importance of considering the impact of algorithmic components on optimisation performance during automated composition, which provides new insights into algorithm design. The thesis gains valuable insights from various learning perspectives, enhancing the understanding towards automated algorithm design. Some directions for future work are present