An investigation of production and transportation policies for multi-item and multi-stage production systems

Die vorliegende kumulative Dissertation besteht aus fünf Artikeln, einem Arbeitspapier und vier Artikeln, die in wissenschaftlichen Zeitschriften veröffentlicht wurden. Alle fünf Artikel beschäftigen sich mit der Losgrößenplanung, jedoch mit unterschiedlichen Schwerpunkten. Artikel 1 bis 4 untersuchen das Economic Lot Scheduling Problem (ELSP), während sich der fünfte Artikel mit einer Variante des Joint Economic Lot Size (JELS) Problems beschäftigt. Die Struktur dieser Dissertation trägt diesen beiden Forschungsrichtungen Rechnung und ordnet die ersten vier Artikel dem Teil A und den fünften Artikel dem Teil B zu

Setup Change Scheduling Under Due-date Constraints Using Deep Reinforcement Learning with Self-supervision

학위논문(박사) -- 서울대학교대학원 : 공과대학 산업·조선공학부, 2021.8. 박종헌.납기 제약 하에서 셋업 스케줄을 수립하는 것은 현실의 여러 제조 산업에서 쉽게 찾아 볼 수 있으며 학계의 많은 관심을 끌고 있는 중대한 문제이다. 그러나 납기와 셋업 제약이 동시에 존재함에 따라 문제의 복잡도가 증가하게 되며, 시시각각 새로운 생산 계획이 주어지고 초기 설비 상태가 변화되는 환경에서 고품질의 스케줄 수립은 더 어려워진다. 본 논문에서는 학습된 심층신경망이 상기한 변화가 발생한 스케줄링 문제도 재학습 없이 해결할 수 있도록, 자기지도 기반 심층강화학습 기법을 제안한다. 구체적으로, 상태와 행동 표현을 생산 계획과 설비 상태에 무관한 차원을 갖도록 설계한다. 동시에 주어진 상태로부터 효율적으로 신경망을 학습하기 위해 파라미터 공유 구조를 도입한다. 이에 더하여, 스케줄링 문제에 적합한 자기지도를 고안하여 설비와 잡의 수, 생산 계획의 분포가 상이한 평가 환경으로도 일반화 가능한 심층신경망을 학습한다. 제안 기법의 유효성을 검증하기 위해 현실의 병렬설비 및 잡샵 공정을 모사한 대규모 데이터셋에서 집약적인 실험을 수행하였다. 제안 기법을 메타휴리스틱 기법과 다른 강화학습 기반 기법, 규칙 기반 기법과 비교함으로써 납기 준수 성능과 연산 시간 관점에서 우수성을 입증하였다. 더불어 상태 표현, 파라미터 공유, 자기지도 각각으로 인한 효과를 조사한 결과, 개별적으로 성능 개선에 기여함을 밝혀냈다.Setup change scheduling under due-date constraints has attracted much attention from academia and industry due to its practical applications. In a real-world manufacturing system, however, solving the scheduling problem becomes challenging since it is required to address urgent and frequent changes in demand and due-dates of products, and initial machine status. In this thesis, we propose a scheduling framework based on deep reinforcement learning (RL) with self-supervision in which trained neural networks (NNs) are able to solve unseen scheduling problems without re-training even when such changes occur. Specifically, we propose state and action representations whose dimensions are independent of production requirements and due-dates of jobs while accommodating family setups. At the same time, an NN architecture with parameter sharing was utilized to improve the training efficiency. Finally, we devise an additional self-supervised loss specific to the scheduling problem for training the NN scheduler robust to the variations in the numbers of machines and jobs, and distribution of production plans. We carried out extensive experiments in large-scale datasets that simulate the real-world wafer preparation facility and semiconductor packaging line. Experiment results demonstrate that the proposed method outperforms the recent metaheuristics, rule-based, and other RL-based methods in terms of the schedule quality and computation time for obtaining a schedule. Besides, we investigated individual contributions of the state representation, parameter sharing, and self-supervision on the performance improvements.제 1 장 서론 1 1.1 연구 동기 및 배경 1 1.2 연구 목적 및 공헌 4 1.3 논문구성 6 제 2 장 배경 7 2.1 순서 의존적 셋업이 있는 납기 제약 하에서의 스케줄링 문제 7 2.1.1 납기 제약 하에서의 스케줄링 문제 7 2.1.2 패밀리 셋업을 고려한 병렬설비 스케줄링 8 2.1.3 셋업 제약이 있는 잡샵 스케줄링 9 2.2 강화학습 기반 스케줄링 12 2.2.1 이론적 배경 12 2.2.2 강화학습을 이용한 제조 라인 스케줄링 13 2.2.3 스케줄링 문제에서의 심층강화학습 15 2.3 자기지도 기반 심층강화학습 19 제 3 장 문제 정의 22 3.1 병렬설비 스케줄링 문제 22 3.1.1 지연시간 최소화를 위한 병렬설비 스케줄링 문제 22 3.1.2 혼합정수계획 모형 24 3.1.3 예시 공정 25 3.2 잡샵 스케줄링 문제 26 3.2.1 투입량 최대화를 위한 유연잡샵 스케줄링 26 3.2.2 예시 공정 27 제 4 장 자기지도 기반 심층강화학습을 이용한 병렬설비 스케줄링 31 4.1 MDP 모형 31 4.1.1 행동 정의 31 4.1.2 상태 표현 32 4.1.3 보상 정의 37 4.1.4 상태 전이 38 4.1.5 예시 39 4.2 신경망 학습 41 4.2.1 심층신경망 구조 41 4.2.2 손실 함수 42 4.2.3 DQN 학습 절차 43 4.2.4 DQN 평가 절차 44 4.3 스케줄링 문제에서의 자기지도 46 4.3.1 내재적 보상 설계 46 4.3.2 셋업 스케줄링을 위한 선호도 점수 설계 47 4.4 자기지도 기반 DQN 학습 49 4.4.1 자기지도 손실 함수 49 4.4.2 학습 절차 50 제 5 장 자기지도 기반 심층강화학습을 이용한 잡샵 스케줄링 53 5.1 스케줄링 프레임워크 53 5.1.1 병목 공정 정의 53 5.1.2 디스패치 규칙 54 5.1.3 이산 사건 시뮬레이터 55 5.1.4 스케줄러 학습 56 5.2 투입 정책과 자기지도 58 5.3 MDP 모형 수정 59 5.3.1 행동 정의 59 5.3.2 상태 표현 59 5.3.3 보상 정의 61 제 6 장 실험 및 결과 62 6.1 병렬설비 스케줄링 문제 62 6.1.1 데이터셋 62 6.1.2 실험 세팅 64 6.1.3 지연시간 총합 성능 비교 67 6.1.4 상태 표현 방식에 따른 성능 비교 72 6.2 잡샵 스케줄링 문제 74 6.2.1 데이터셋 74 6.2.2 실험 세팅 75 6.2.3 투입량 성능 비교 77 6.2.4 행동 정의 방식에 따른 성능 비교 80 6.3 자기지도로 인한 효과 84 6.3.1 데이터셋 84 6.3.2 실험 세팅 86 6.3.3 파라미터 공유 여부에 따른 자기지도의 효과 87 6.3.4 학습 시와 다른 데이터셋에서의 성능 평가 91 제 7 장 결론 및 향후 연구 방향 96 7.1 결론 96 7.2 향후 연구 방향 98 참고문헌 100 Abstract 118 감사의 글 120박

Hybrid Meta-heuristic Algorithms for Static and Dynamic Job Scheduling in Grid Computing

The term ’grid computing’ is used to describe an infrastructure that connects geographically distributed computers and heterogeneous platforms owned by multiple organizations allowing their computational power, storage capabilities and other resources to be selected and shared. Allocating jobs to computational grid resources in an efficient manner is one of the main challenges facing any grid computing system; this allocation is called job scheduling in grid computing. This thesis studies the application of hybrid meta-heuristics to the job scheduling problem in grid computing, which is recognized as being one of the most important and challenging issues in grid computing environments. Similar to job scheduling in traditional computing systems, this allocation is known to be an NPhard problem. Meta-heuristic approaches such as the Genetic Algorithm (GA), Variable Neighbourhood Search (VNS) and Ant Colony Optimisation (ACO) have all proven their effectiveness in solving different scheduling problems. However, hybridising two or more meta-heuristics shows better performance than applying a stand-alone approach. The new high level meta-heuristic will inherit the best features of the hybridised algorithms, increasing the chances of skipping away from local minima, and hence enhancing the overall performance. In this thesis, the application of VNS for the job scheduling problem in grid computing is introduced. Four new neighbourhood structures, together with a modified local search, are proposed. The proposed VNS is hybridised using two meta-heuristic methods, namely GA and ACO, in loosely and strongly coupled fashions, yielding four new sequential hybrid meta-heuristic algorithms for the problem of static and dynamic single-objective independent batch job scheduling in grid computing. For the static version of the problem, several experiments were carried out to analyse the performance of the proposed schedulers in terms of minimising the makespan using well known benchmarks. The experiments show that the proposed schedulers achieved impressive results compared to other traditional, heuristic and meta-heuristic approaches selected from the bibliography. To model the dynamic version of the problem, a simple simulator, which uses the rescheduling technique, is designed and new problem instances are generated, by using a well-known methodology, to evaluate the performance of the proposed hybrid schedulers. The experimental results show that the use of rescheduling provides significant improvements in terms of the makespan compared to other non-rescheduling approaches

Data-Intensive Computing in Smart Microgrids

Microgrids have recently emerged as the building block of a smart grid, combining distributed renewable energy sources, energy storage devices, and load management in order to improve power system reliability, enhance sustainable development, and reduce carbon emissions. At the same time, rapid advancements in sensor and metering technologies, wireless and network communication, as well as cloud and fog computing are leading to the collection and accumulation of large amounts of data (e.g., device status data, energy generation data, consumption data). The application of big data analysis techniques (e.g., forecasting, classification, clustering) on such data can optimize the power generation and operation in real time by accurately predicting electricity demands, discovering electricity consumption patterns, and developing dynamic pricing mechanisms. An efficient and intelligent analysis of the data will enable smart microgrids to detect and recover from failures quickly, respond to electricity demand swiftly, supply more reliable and economical energy, and enable customers to have more control over their energy use. Overall, data-intensive analytics can provide effective and efficient decision support for all of the producers, operators, customers, and regulators in smart microgrids, in order to achieve holistic smart energy management, including energy generation, transmission, distribution, and demand-side management. This book contains an assortment of relevant novel research contributions that provide real-world applications of data-intensive analytics in smart grids and contribute to the dissemination of new ideas in this area

Mejoramiento de programación de producción en planta de inyección de plásticos usando un algoritmo genético

70 páginasEste proyecto de grado buscó solucionar el problema de secuenciar un conjunto de trabajos y moldes de inyección en máquinas inyectoras en una planta de inyección, con el fin de reducir el makespan y tardanza. La planta de inyección es el primer eslabón de la cadena de producción en una fábrica de productos de consumo masivo. El proyecto se caracterizó como un problema de secuenciación de trabajos en máquinas no relacionadas paralelas. El proyecto aborda el problema con dos métodos, el primero usando programación lineal entera mixta (MILP) y el segundo usando un algoritmo genético. El método exacto funciona bien con instancias pequeñas de máximo 10 trabajos y cinco máquinas. Respecto al segundo método se diseñó un algoritmo genético con una función fitness completamente original, el algoritmo genético permite encontrar soluciones de calidad en corto tiempo para instancias más grandes y complejas si se compara con el método exacto. El algoritmo genético requirió un ajuste de sus parámetros usando diseño factorial multinivel. Con el objetivo de probar el método exacto de solución y lograr una comparación estricta entre los dos métodos de solución se desarrollaron instancias de dos tipos: random y reales con información de la planta de inyección. Luego se desarrollaron experimentos computacionales solucionando las instancias con los dos métodos. Los resultados de los experimentos permitieron establecer que el algoritmo genético propuesto genera soluciones iguales o mejores en instancias random comparado con el método exacto.This thesis developed a method to find solutions to a scheduling problem in an injection mold factory, intending to reduce the makespan and tardiness. The injection mould factory is the first stage in a massive consumer product factory location. The project was characterized as a nonrelated parallel machine scheduling problem. The project approached the problem with two methods: the 1st one using mixed-integer linear programming (MILP) and the 2nd one using a genetic algorithm. The exact solution method works fine with small instances, with a maximal size of ten jobs and five machines. About the second method, a genetic algorithm was designed with a completely original fitness function, the genetic algorithm was able to find several quality solutions in a shorter time for larger and complex instances if it is compared with the exact solution method. The genetic algorithm required several parameter adjustments, the multilevel factorial design was used to do so. With the objective of testing the exact method and to can achieve a strict comparison between both solution methods, two kinds of instances were developed: 1 kind with random data and the other one with real production data. After it, several computational experiments were developed, solving the whole instances with both solution methods. Experiments results allowed the researchers to conclude that the proposed genetic algorithm creates equal or better solutions if it is compared with the exact solution method with random instances. One study case was developed, this case represents the production schedule for a month for injected components labeled as type A, the study case was used to compare the current scheduling method used in the injection factory versus the proposed genetic algorithm. The genetic algorithm provides a scheduling program with Cmax and Tardiness values 30% lower than the current scheduling method.Maestría en Gerencia de IngenieríaMagíster en Gerencia de Ingenierí

LIPIcs, Volume 244, ESA 2022, Complete Volume

LIPIcs, Volume 244, ESA 2022, Complete Volum

Improvements on the bees algorithm for continuous optimisation problems

This work focuses on the improvements of the Bees Algorithm in order to enhance the algorithm’s performance especially in terms of convergence rate. For the first enhancement, a pseudo-gradient Bees Algorithm (PG-BA) compares the fitness as well as the position of previous and current bees so that the best bees in each patch are appropriately guided towards a better search direction after each consecutive cycle. This method eliminates the need to differentiate the objective function which is unlike the typical gradient search method. The improved algorithm is subjected to several numerical benchmark test functions as well as the training of neural network. The results from the experiments are then compared to the standard variant of the Bees Algorithm and other swarm intelligence procedures. The data analysis generally confirmed that the PG-BA is effective at speeding up the convergence time to optimum. Next, an approach to avoid the formation of overlapping patches is proposed. The Patch Overlap Avoidance Bees Algorithm (POA-BA) is designed to avoid redundancy in search area especially if the site is deemed unprofitable. This method is quite similar to Tabu Search (TS) with the POA-BA forbids the exact exploitation of previously visited solutions along with their corresponding neighbourhood. Patches are not allowed to intersect not just in the next generation but also in the current cycle. This reduces the number of patches materialise in the same peak (maximisation) or valley (minimisation) which ensures a thorough search of the problem landscape as bees are distributed around the scaled down area. The same benchmark problems as PG-BA were applied against this modified strategy to a reasonable success. Finally, the Bees Algorithm is revised to have the capability of locating all of the global optimum as well as the substantial local peaks in a single run. These multi-solutions of comparable fitness offers some alternatives for the decision makers to choose from. The patches are formed only if the bees are the fittest from different peaks by using a hill-valley mechanism in this so called Extended Bees Algorithm (EBA). This permits the maintenance of diversified solutions throughout the search process in addition to minimising the chances of getting trap. This version is proven beneficial when tested with numerous multimodal optimisation problems

Adaptive and Scalable Controller Placement in Software-Defined Networking

Software-defined networking (SDN) revolutionizes network control by externalizing and centralizing the control plane. A critical aspect of SDN is Controller Placement (CP), which involves identifying the ideal number and location of controllers in a network to fulfill diverse objectives such as latency constraints (node-to-controller and controller-controller delay), fault tolerance, and controller load. Existing optimization techniques like Multi-Objective Particle Swarm Optimisation (MOPSO), Adapted Non-Dominating Sorting Genetic Algorithm-III (ANSGA-III), and Non-Dominating Sorting Genetic Algorithm-II (NSGA-II) struggle with scalability (except ANSGA-III), computational complexity, and inability to predict the required number of controllers. This thesis proposes two novel approaches to address these challenges. First, an enhanced version of NSGA-III with a repair operator-based approach (referred to as ANSGA-III) is introduced, enabling efficient CP in SD-WAN by optimizing multiple conflicting objectives simultaneously. Second, a Stochastic Computational Graph Model with Ensemble Learning (SCGMEL) is developed, overcoming scalability and computational inefficiency associated with existing methods. SCGMEL employs stochastic gradient descent with momentum, a learning rate decay, a computational graph model, a weighted sum approach, and the XGBoost algorithm for optimization and machine learning. The XGBoost predicts the number of controllers needed and a supervised classification algorithm called Learning Vector Quantization (LVQ) is used to predict the optimal locations of controllers. Additionally, this research introduces the Improved Switch Migration Decision Algorithm (ISMDA) as part of the holistic contribution. ISMDA is implemented on each controller to ensure even load distribution throughout the controllers. It functions as a plug-and-play module, periodically checking if the load surpasses a certain limit. ISMDA improves controller throughput by approximately 7.4% over CAMD and roughly 1.1% over DALB. ISMDA also outperforms DALB and CAMD with a decrease of 5.7% and 1%, respectively, in terms of controller response time. Additionally, ISMDA outperforms DALB and CAMD with a decrease of 1.7% and 5.6%, respectively, in terms of the average frequency of migrations. The established framework results in fewer switch migrations during controller load imbalance. Finally, ISMDA proves more efficient than DALB and CAMD, with an estimated 1% and 6.4% lower average packet loss, respectively. This efficiency is a result of the proposed migration efficiency strategy, allowing ISMDA to handle higher loads and reject fewer packets. Real-world experiments were conducted using the Internet Zoo topology dataset to evaluate the proposed solutions. Six objective functions, including worst-case switch-to-controller delay, load balancing, reliability, average controller-to-controller latency, maximum controller-to-controller delay, and average switch-to-controller delay, were utilized for performance evaluation. Results demonstrated that ANSGA-III outperforms existing algorithms in terms of hypervolume indicator, execution time, convergence, diversity, and scalability. SCGMEL exhibited exceptional computational efficiency, surpassing ANSGA-III, NSGA-II, and MOPSO by 99.983%, 99.985%, and 99.446% respectively. The XGBoost regression model performed significantly better in predicting the number of controllers with a mean absolute error of 1.855751 compared to 3.829268, 3.729883, and 1.883536 for KNN, linear regression, and random forest, respectively. The proposed LVQ-based classification method achieved a test accuracy of 84% and accurately predicted six of the seven controller locations. To culminate, this study presents a refined and intelligent framework designed to optimize Controller Placement (CP) within the context of SD-WAN. The proposed solutions effectively tackle the shortcomings associated with existing algorithms, addressing challenges of scalability, intelligence (including the prediction of optimal controller numbers), and computational efficiency in the pursuit of simultaneous optimization of multiple conflicting objectives. The outcomes underscore the supremacy of the suggested methodologies and underscore their potential transformative influence on SDN deployments. Notably, the findings validate the efficacy of the proposed strategies, ANSGA-III and SCGMEL, in enhancing the optimization of controller placement within SD-WAN setups. The integration of the XGBoost regression model and LVQ-based classification technique yields precise predictions for both optimal controller quantities and their respective positions. Additionally, the ISMDA algorithm emerges as a pivotal enhancement, enhancing controller throughput, mitigating packet losses, and reducing switch migration frequency—collectively contributing to elevated standards in SDN deployments