A Memetic Algorithm with Reinforcement Learning for Sociotechnical Production Scheduling

The following interdisciplinary article presents a memetic algorithm with applying deep reinforcement learning (DRL) for solving practically oriented dual resource constrained flexible job shop scheduling problems (DRC-FJSSP). From research projects in industry, we recognize the need to consider flexible machines, flexible human workers, worker capabilities, setup and processing operations, material arrival times, complex job paths with parallel tasks for bill of material (BOM) manufacturing, sequence-dependent setup times and (partially) automated tasks in human-machine-collaboration. In recent years, there has been extensive research on metaheuristics and DRL techniques but focused on simple scheduling environments. However, there are few approaches combining metaheuristics and DRL to generate schedules more reliably and efficiently. In this paper, we first formulate a DRC-FJSSP to map complex industry requirements beyond traditional job shop models. Then we propose a scheduling framework integrating a discrete event simulation (DES) for schedule evaluation, considering parallel computing and multicriteria optimization. Here, a memetic algorithm is enriched with DRL to improve sequencing and assignment decisions. Through numerical experiments with real-world production data, we confirm that the framework generates feasible schedules efficiently and reliably for a balanced optimization of makespan (MS) and total tardiness (TT). Utilizing DRL instead of random metaheuristic operations leads to better results in fewer algorithm iterations and outperforms traditional approaches in such complex environments.Comment: This article has been accepted by IEEE Access on June 30, 202

Reinforcement Learning-assisted Evolutionary Algorithm: A Survey and Research Opportunities

Evolutionary algorithms (EA), a class of stochastic search methods based on the principles of natural evolution, have received widespread acclaim for their exceptional performance in various real-world optimization problems. While researchers worldwide have proposed a wide variety of EAs, certain limitations remain, such as slow convergence speed and poor generalization capabilities. Consequently, numerous scholars actively explore improvements to algorithmic structures, operators, search patterns, etc., to enhance their optimization performance. Reinforcement learning (RL) integrated as a component in the EA framework has demonstrated superior performance in recent years. This paper presents a comprehensive survey on integrating reinforcement learning into the evolutionary algorithm, referred to as reinforcement learning-assisted evolutionary algorithm (RL-EA). We begin with the conceptual outlines of reinforcement learning and the evolutionary algorithm. We then provide a taxonomy of RL-EA. Subsequently, we discuss the RL-EA integration method, the RL-assisted strategy adopted by RL-EA, and its applications according to the existing literature. The RL-assisted procedure is divided according to the implemented functions including solution generation, learnable objective function, algorithm/operator/sub-population selection, parameter adaptation, and other strategies. Finally, we analyze potential directions for future research. This survey serves as a rich resource for researchers interested in RL-EA as it overviews the current state-of-the-art and highlights the associated challenges. By leveraging this survey, readers can swiftly gain insights into RL-EA to develop efficient algorithms, thereby fostering further advancements in this emerging field.Comment: 26 pages, 16 figure

Two-Stage Adaptive Memetic Algorithm with Surprisingly Popular Mechanism for Energy-Aware Distributed Hybrid Flow Shop Scheduling Problem with Sequence-Dependent Setup Time

This paper considers the impact of setup time in production scheduling and proposes energy-aware distributed hybrid flow shop scheduling problem with sequence-dependent setup time (EADHFSP-ST) that simultaneously optimizes the makespan and the energy consumption. We develop a mixed integer linear programming model to describe this problem and present a two-stage adaptive memetic algorithm (TAMA) with a surprisingly popular mechanism. First, a hybrid initialization strategy is designed based on the two optimization objectives to ensure the convergence and diversity of solutions. Second, multiple population co-evolutionary approaches are proposed for global search to escape from traditional cross-randomization and to balance exploration and exploitation. Third, considering that the memetic algorithm (MA) framework is less efficient due to the randomness in the selection of local search operators, TAMA is proposed to balance the local and global searches. The first stage accumulates more experience for updating the surprisingly popular algorithm (SPA) model to guide the second stage operator selection and ensures population convergence. The second stage gets rid of local optimization and designs an elite archive to ensure population diversity. Fourth, five problem-specific operators are designed, and non-critical path deceleration and right-shift strategies are designed for energy efficiency. Finally, to evaluate the performance of the proposed algorithm, multiple experiments are performed on a benchmark with 45 instances. The experimental results show that the proposed TAMA can solve the problem effectively

A survey of scheduling problems with setup times or costs

Author name used in this publication: C. T. NgAuthor name used in this publication: T. C. E. Cheng2007-2008 > Academic research: refereed > Publication in refereed journalAccepted ManuscriptPublishe

Hybrid Job Shop and Parallel Machine Scheduling Problems: Minimization of Total Tardiness Criterion

International audienc

Energy Efficient Policies, Scheduling, and Design for Sustainable Manufacturing Systems

Climate mitigation, more stringent regulations, rising energy costs, and sustainable manufacturing are pushing researchers to focus on energy efficiency, energy flexibility, and implementation of renewable energy sources in manufacturing systems. This thesis aims to analyze the main works proposed regarding these hot topics, and to fill the gaps in the literature. First, a detailed literature review is proposed. Works regarding energy efficiency in different manufacturing levels, in the assembly line, energy saving policies, and the implementation of renewable energy sources are analyzed. Then, trying to fill the gaps in the literature, different topics are analyzed more in depth. In the single machine context, a mathematical model aiming to align the manufacturing power required to a renewable energy supply in order to obtain the maximum profit is developed. The model is applied to a single work center powered by the electric grid and by a photovoltaic system; afterwards, energy storage is also added to the power system. Analyzing the job shop context, switch off policies implementing workload approach and scheduling considering variable speed of the machines and power constraints are proposed. The direct and indirect workloads of the machines are considered to support the switch on/off decisions. A simulation model is developed to test the proposed policies compared to others presented in the literature. Regarding the job shop scheduling, a fixed and variable power constraints are considered, assuming the minimization of the makespan as the objective function. Studying the factory level, a mathematical model to design a flow line considering the possibility of using switch-off policies is developed. The design model for production lines includes a targeted imbalance among the workstations to allow for defined idle time. Finally, the main findings, results, and the future directions and challenges are presented

Design of a Reference Architecture for Production Scheduling Applications based on a Problem Representation including Practical Constraints

Changing customer demands increase the complexity and importance of production scheduling, requiring better scheduling algorithms, e.g., machine learning algorithms. At the same time, current research often neglects practical constraints, e.g., changeovers or transportation. To address this issue, we derive a representation of the scheduling problem and develop a reference architecture for future scheduling applications to increase the impact of future research. To achieve this goal, we apply a design science research approach and, first, rigorously identify the problem and derive requirements for a scheduling application based on a structured literature review. Then, we develop the problem representation and reference architecture as design science artifacts. Finally, we demonstrate the artifacts in an application scenario and publish the resulting prototypical scheduling application, enabling machine learning-based scheduling algorithms, for usage in future development projects. Our results guide future research into including practical constraints and provide practitioners with a framework for developing scheduling applications