Genetic Programming Hyper-heuristics for Dynamic Flexible Job Shop Scheduling

Dynamic flexible job shop scheduling (DFJSS) has received widespread attention from academia and industry due to its reflection in real-world scheduling applications such as order picking in the warehouse and the manufacturing industry. It requires complex routing and sequencing decisions under unpredicted dynamic events. Genetic programming, as ahyper-heuristic approach (GPHH), has been successfully applied to evolve scheduling heuristics for DFJSS automatically due to its flexible representation. Although GPHH has achieved certain success in solving the DFJSS problems, there are still some limitations for applying GPHH to DFJSS, particularly in terms of its training efficiency, large search space, search mechanism, and multitask solving ability. The overall goal of this thesis is to develop effective GPHH algorithmsto evolve scheduling heuristics for DFJSS efficiently. Different machine learning techniques, i.e., surrogate, feature selection, specialised genetic operator, and multitask learning, are incorporated in this thesis to tackle the limitations. First, this thesis develops a novel multi-fidelity based surrogate-assisted GPHH for DFJSS to improve the training efficiency of GPHH. Specifically, multi-fidelity based surrogate models are first designed by simplifying the problem to be solved. Then, an effective collaboration mechanism with knowledge transfer is proposed for utilising the advantages of the multifidelity based surrogate models to solve the problem. The results show that the proposed algorithm can dramatically reduce the computational cost of GPHH without sacrificing the performance in all the test scenarios. With the same training time, the proposed algorithm can achieve significantly better performance than its counterparts in most scenarios while no worse in others. Second, this thesis designs a novel two-stage GPHH framework with feature selection to evolve scheduling heuristics for DFJSS automatically. Based on this framework, this thesis further proposes to evolve scheduling heuristics with only the selected features by eliminating the unselected features properly. Specifically, individual adaptation strategies are proposed to generate individuals with only the selected features by utilising the information of both the selected features and the investigated individuals during the feature selection process. The results show that the proposed algorithm can successfully achieve scheduling heuristics with fewer unique features and smaller sizes, which tends to be more interpretable. In addition, the proposed algorithm can evolve comparable scheduling heuristic with that obtained by the traditional GPHH within a much shorter training time. Third, this thesis proposes a novel recombinative mechanism to provide guidance for GPHH based on the importance of subtrees to realise effective and adaptive recombination for parents to produce offspring. Two measures are proposed to measure the importance of all the subtrees of an individual. The first one is based on the frequency of features, and the second is based on the correlation between the behaviour of subtrees and the whole tree (i.e., an individual). The importance information is utilised to decide the crossover points for the parents. The proposed recombinative guidance mechanism attempts to improve the quality of offspring by preserving the promising building-blocks of one parent and incorporating good building-blocks from the other. The results show that the proposed algorithm based on the correlation importance measure performs better than the proposed algorithm based on the feature frequency importance measure. In addition, the proposed algorithm based on the correlation importance measure between the behaviour of subtrees significantly also outperforms the state-of-the-art algorithms on most tested scenarios. Last, this thesis proposes a multitask GPHH approach and a surrogate-assisted multitask GPHH approach to solving multiple DFJSS tasks simultaneously. First, an effective hyper-heuristic multitask algorithm is proposed by adapting the traditional evolutionary multitask algorithms based on the characteristics of GPHH. Second, this thesis develops a novel surrogate-assisted multitask GPHH approach to solving multiple DFJSS tasks by sharing useful knowledge between different DFJSS scheduling tasks. Specifically, the surrogate-assisted multitask GPHH algorithm employs the phenotypic characterisation technique to measure the behaviours of scheduling rules to build a surrogate for each task accordingly. The built surrogates are not only used to improve the efficiency of solving each single DFJSS task but also utilised for knowledge sharing between multiple DFJSS tasks in multitask learning. The results show that the proposed algorithm can significantly improve the quality of scheduling heuristics for all the test scenarios. The results also observe that the proposed algorithms manage to solve multiple tasks collaboratively in terms of the evolved scheduling heuristics for different tasks in a multitask scenario.</p

Toward Evolving Dispatching Rules With Flow Control Operations By Grammar-Guided Linear Genetic Programming

Linear genetic programming (LGP) has been successfully applied to dynamic job shop scheduling (DJSS) to automatically evolve dispatching rules. Flow control operations are crucial in concisely describing complex knowledge of dispatching rules, such as different dispatching rules in different conditions. However, existing LGP methods for DJSS have not fully considered the use of flow control operations. They simply included flow control operations in their primitive set, which inevitably leads to a huge number of redundant and obscure solutions in LGP search spaces. To move one step toward evolving effective and interpretable dispatching rules, this paper explicitly considers the characteristics of flow control operations via grammar-guided linear genetic programming and focuses on IF operations as a starting point. Specifically, this paper designs a new set of normalized terminals to improve the interpretability of IF operations and proposes three restrictions by grammar rules on the usage of IF operations: specifying the available inputs, the maximum number, and the possible locations of IF operations. The experiment results verify that the proposed method can achieve significantly better test performance than state-of-the-art LGP methods and improves interpretability by IF-included dispatching rules. Further investigation confirms that the explicit introduction of IF operations helps effectively evolve different dispatching rules according to their decision situations

Correlation Coefficient-Based Recombinative Guidance for Genetic Programming Hyperheuristics in Dynamic Flexible Job Shop Scheduling

Dynamic flexible job shop scheduling (JSS) is a challenging combinatorial optimization problem due to its complex environment. In this problem, machine assignment and operation sequencing decisions need to be made simultaneously under the dynamic environments. Genetic programming (GP), as a hyperheuristic approach, has been successfully used to evolve scheduling heuristics for dynamic flexible JSS. However, in traditional GP, recombination between parents may disrupt the beneficial building blocks by choosing the crossover points randomly. This article proposes a recombinative mechanism to provide guidance for GP to realize effective and adaptive recombination for parents to produce offspring. Specifically, we define a novel measure for the importance of each subtree of an individual, and the importance information is utilized to decide the crossover points. The proposed recombinative guidance mechanism attempts to improve the quality of offspring by preserving the promising building blocks of one parent and incorporating good building blocks from the other. The proposed algorithm is examined on six scenarios with different configurations. The results show that the proposed algorithm significantly outperforms the state-of-the-art algorithms on most tested scenarios, in terms of both final test performance and convergence speed. In addition, the rules obtained by the proposed algorithm have good interpretability

Evolving Scheduling Heuristics via Genetic Programming with Feature Selection in Dynamic Flexible Job Shop Scheduling

No description supplie

Genetic Programming and Reinforcement Learning on Learning Heuristics for Dynamic Scheduling: A Preliminary Comparison

Scheduling heuristics are commonly used to solve dynamic scheduling problems in real-world applications. However, designing effective heuristics can be time-consuming and often leads to suboptimal performance. Genetic programming has been widely used to automatically learn scheduling heuristics. In recent years, reinforcement learning has also gained attention in this field. Understanding their strengths and weaknesses is crucial for developing effective scheduling heuristics. This paper takes a typical genetic programming method and a typical reinforcement learning method in dynamic flexible job shop scheduling for investigation. The results show that the investigated genetic programming algorithm outperforms the studied reinforcement learning method in the examined scenarios. Also, the study reveals that the compared reinforcement learning method is more stable as the amount of training data changes, and the investigated genetic programming method can learn more effective scheduling heuristics as training data increases. Additionally, the study highlights the potential and value of genetic programming in real-world applications due to its good generalization ability and interpretability. Based on the results, this paper suggests using the investigated reinforcement learning method when training data is limited and stable results are required, and using the investigated genetic programming method when training data is sufficient and high interpretability is required

Niching Genetic Programming to Learn Actions for Deep Reinforcement Learning in Dynamic Flexible Scheduling

Dynamic Flexible Job Shop Scheduling (DFJSS) is a critical combinatorial optimisation problem known for its dynamic nature and flexibility of machines. Traditional scheduling methods face limitations in adapting to such dynamic and flexible environments. Recently, there has been a trend in employing reinforcement learning (RL) to train scheduling agents for selecting manual scheduling heuristics at various decision points for DFJSS. However, the effectiveness of RL is constrained by the limited efficacy of the manually designed scheduling heuristics. Additionally, the process of manually designing diverse scheduling heuristics as the actions demands significant expert knowledge. In response, this paper proposes a Niching genetic programming (GP)-assisted RL method that leverages the evolutionary capabilities of GP to help RL solve the DFJSS problem effectively. Specifically, instead of using those manual scheduling heuristics, the RL actions are replaced with scheduling heuristics evolved by the Niching GP to optimise and adapt these heuristics based on real-time feedback from the environment. Experimental results demonstrate the effectiveness of the proposed method in comparison to the widely used manual scheduling heuristics and the baseline deep RL method. Further analyses reveal that the effectiveness of the proposed method is due to the behavioral differences among heuristics learned by the Niching GP, serving as actions for the RL. In addition, the effectiveness of the proposed algorithm benefits from the comparable percentages of contributions made by these learned heuristics throughout the long-term scheduling process

Collaborative Multifidelity-Based Surrogate Models for Genetic Programming in Dynamic Flexible Job Shop Scheduling

Dynamic flexible job shop scheduling (JSS) has received widespread attention from academia and industry due to its practical application value. It requires complex routing and sequencing decisions under unpredicted dynamic events. Genetic programming (GP), as a hyperheuristic approach, has been successfully applied to evolve scheduling heuristics for JSS due to its flexible representation. However, the simulation-based evaluation is computationally expensive since there are many calculations based on individuals for making decisions in the simulation. To improve training efficiency, this article proposes a novel multifidelity-based surrogate-assisted GP. Specifically, multifidelity-based surrogate models are first designed by simplifying the problem expected to be solved. In addition, this article proposes an effective collaboration mechanism with knowledge transfer for utilizing the advantages of multifidelity-based surrogate models to solve the desired problems. This article examines the proposed algorithm in six different scenarios. The results show that the proposed algorithm can dramatically reduce the computational cost of GP without sacrificing the performance in all scenarios. With the same training time, the proposed algorithm can achieve significantly better performance than its counterparts in most scenarios while no worse in others

Bridging directed acyclic graphs to linear representations in linear genetic programming: a case study of dynamic scheduling

Linear genetic programming (LGP) is a genetic programming paradigm based on a linear sequence of instructions being executed. An LGP individual can be decoded into a directed acyclic graph. The graph intuitively reflects the primitives and their connection. However, existing studies on LGP miss an important aspect when seeing LGP individuals as graphs, that is, the reverse transformation from graph to LGP genotype. Such reverse transformation is an essential step if one wants to use other graph-based techniques and applications with LGP. Transforming graphs into LGP genotypes is nontrivial since graph information normally does not convey register information, a crucial element in LGP individuals. Here we investigate the effectiveness of four possible transformation methods based on different graph information including frequency of graph primitives, adjacency matrices, adjacency lists, and LGP instructions for sub-graphs. For each transformation method, we design a corresponding graph-based genetic operator to explicitly transform LGP parent’s instructions to graph information, then to the instructions of offspring resulting from breeding on graphs. We hypothesize that the effectiveness of the graph-based operators in evolution reflects the effectiveness of different graph-to-LGP genotype transformations. We conduct the investigation by a case study that applies LGP to design heuristics for dynamic scheduling problems. The results show that highlighting graph information improves LGP average performance for solving dynamic scheduling problems. This shows that reversely transforming graphs into LGP instructions based on adjacency lists is an effective way to maintain both primitive frequency and topological structures of graphs

Surrogate-Assisted Evolutionary Multitask Genetic Programming for Dynamic Flexible Job Shop Scheduling

Dynamic flexible job shop scheduling (JSS) is an important combinatorial optimization problem with complex routing and sequencing decisions under dynamic environments. Genetic programming (GP), as a hyperheuristic approach, has been successfully applied to evolve scheduling heuristics for JSS. However, its training process is time consuming, and it faces the retraining problem once the characteristics of job shop scenarios vary. It is known that multitask learning is a promising paradigm for solving multiple tasks simultaneously by sharing knowledge among the tasks. To improve the training efficiency and effectiveness, this article proposes a novel surrogate-assisted evolutionary multitask algorithm via GP to share useful knowledge between different scheduling tasks. Specifically, we employ the phenotypic characterization for measuring the behaviors of scheduling rules and building a surrogate for each task accordingly. The built surrogates are used not only to improve the efficiency of solving each single task but also for knowledge transfer in multitask learning with a large number of promising individuals. The results show that the proposed algorithm can significantly improve the quality of scheduling heuristics for all scenarios. In addition, the proposed algorithm manages to solve multiple tasks collaboratively in terms of the evolved scheduling heuristics for different tasks in a multitask scenario

Learning emergency medical dispatch policies via genetic programming

Of great value to modern municipalities is the task of emergency medical response in the community. Resource allocation is vital to ensure minimal response times, which we may perform via human experts or automate by maximising ambulance coverage. To combat black-box modelling, we propose a modularised Genetic Programming Hyper Heuristic framework to learn the five key decisions of Emergency Medical Dispatch (EMD) within a reactive decision-making process. We minimise the representational distance between our work and reality by working with our local ambulance service to design a set of heuristics approximating their current decision-making processes and a set of synthetic datasets influenced by existing patterns in practice. Through our modularised framework, we learn each decision independently to identify those most valuable to EMD and learn all five decisions simultaneously, improving performance by 69% on the largest novel dataset. We analyse the decision-making logic behind several learned rules to further improve our understanding of EMD. For example, we find that emergency urgency is not necessarily considered when dispatching idle ambulances in favour of maximising fleet availability