Approximation schemes for scheduling on a single machine subject to cumulative deterioration and maintenance

We consider a scheduling problem on a single machine to minimize the makespan. The processing conditions are subject to cumulative deterioration, but can be restored by a single maintenance. We link the problem to the Subset-sum problem (if the duration of maintenance is constant) and to the Half-Product Problem (if the duration of maintenance depends on its start time). For both versions of the problem, we adapt the existing fully polynomial-time approximation schemes to our problems by handling the additive constants

Single machine scheduling with a generalized job-dependent cumulative effect

We consider a single machine scheduling problem with changing processing times. The processing conditions are subject to a general cumulative effect, in which the processing time of a job depends on the sum of certain parameters associated with previously scheduled jobs. In previous papers, these parameters are assumed to be equal to the normal processing times of jobs, which seriously limits the practical application of this model. We further generalize this model by allowing every job to respond differently to these cumulative effects. For the introduced model, we solve the problem of minimizing the makespan, with and without precedence constraints. For the problem without precedence constraints, we also consider a situation in which a maintenance activity is included in the schedule, which can improve the processing conditions of the machine, not necessarily to its original state. The resulting problem is reformulated as a variant of a Boolean programming problem with a quadratic objective, known as a half-product, which allows us to develop a fully polynomial-time approximation scheme with the best possible running time

Serial-batch scheduling – the special case of laser-cutting machines

The dissertation deals with a problem in the field of short-term production planning, namely the scheduling of laser-cutting machines. The object of decision is the grouping of production orders (batching) and the sequencing of these order groups on one or more machines (scheduling). This problem is also known in the literature as "batch scheduling problem" and belongs to the class of combinatorial optimization problems due to the interdependencies between the batching and the scheduling decisions. The concepts and methods used are mainly from production planning, operations research and machine learning

A study in joint maintenance scheduling and production planning

Master'sMASTER OF ENGINEERIN

Optimal maintenance of multi-component systems: a review

In this article we give an overview of the literature on multi-component maintenance optimization. We focus on work appearing since the 1991 survey "A survey of maintenance models for multi-unit systems" by Cho and Parlar. This paper builds forth on the review article by Dekker et al. (1996), which focusses on economic dependence, and the survey of maintenance policies by Wang (2002), in which some group maintenance and some opportunistic maintenance policies are considered. Our classification scheme is primarily based on the dependence between components (stochastic, structural or economic). Next, we also classify the papers on the basis of the planning aspect (short-term vs long-term), the grouping of maintenance activities (either grouping preventive or corrective maintenance, or opportunistic grouping) and the optimization approach used (heuristic, policy classes or exact algorithms). Finally, we pay attention to the applications of the models.literature review;economic dependence;failure interaction;maintenance policies;grouping maintenance;multi-component systems;opportunistic maintenance;maintencance optimization;structural dependence

Risk-Based Optimal Scheduling for the Predictive Maintenance of Railway Infrastructure

In this thesis a risk-based decision support system to schedule the predictive maintenance activities, is proposed. The model deals with the maintenance planning of a railway infrastructure in which the due-dates are defined via failure risk analysis.The novelty of the approach consists of the risk concept introduction in railway maintenance scheduling, according to ISO 55000 guidelines, thus implying that the maintenance priorities are based on asset criticality, determined taking into account the relevant failure probability, related to asset degradation conditions, and the consequent damages

모델기반강화학습을이용한공정제어및최적화

학위논문(박사)--서울대학교 대학원 :공과대학 화학생물공학부,2020. 2. 이종민.순차적 의사결정 문제는 공정 최적화의 핵심 분야 중 하나이다. 이 문제의 수치적 해법 중 가장 많이 사용되는 것은 순방향으로 작동하는 직접법 (direct optimization) 방법이지만, 몇가지 한계점을 지니고 있다. 최적해는 open-loop의 형태를 지니고 있으며, 불확정성이 존재할때 방법론의 수치적 복잡도가 증가한다는 것이다. 동적 계획법 (dynamic programming) 은 이러한 한계점을 근원적으로 해결할 수 있지만, 그동안 공정 최적화에 적극적으로 고려되지 않았던 이유는 동적 계획법의 결과로 얻어진 편미분 방정식 문제가 유한차원 벡터공간이 아닌 무한차원의 함수공간에서 다루어지기 때문이다. 소위 차원의 저주라고 불리는 이 문제를 해결하기 위한 한가지 방법으로서, 샘플을 이용한 근사적 해법에 초점을 둔 강화학습 방법론이 연구되어 왔다. 본 학위논문에서는 강화학습 방법론 중, 공정 최적화에 적합한 모델 기반 강화학습에 대해 연구하고, 이를 공정 최적화의 대표적인 세가지 순차적 의사결정 문제인 스케줄링, 상위단계 최적화, 하위단계 제어에 적용하는 것을 목표로 한다. 이 문제들은 각각 부분관측 마르코프 결정 과정 (partially observable Markov decision process), 제어-아핀 상태공간 모델 (control-affine state space model), 일반적 상태공간 모델 (general state space model)로 모델링된다. 또한 각 수치적 모델들을 해결하기 위해 point based value iteration (PBVI), globalized dual heuristic programming (GDHP), and differential dynamic programming (DDP)로 불리는 방법들을 도입하였다. 이 세가지 문제와 방법론에서 제시된 특징들을 다음과 같이 요약할 수 있다: 첫번째로, 스케줄링 문제에서 closed-loop 피드백 형태의 해를 제시할 수 있었다. 이는 기존 직접법에서 얻을 수 없었던 형태로서, 강화학습의 강점을 부각할 수 있는 측면이라 생각할 수 있다. 두번째로 고려한 하위단계 제어 문제에서, 동적 계획법의 무한차원 함수공간 최적화 문제를 함수 근사 방법을 통해 유한차원 벡터공간 최적화 문제로 완화할 수 있는 방법을 도입하였다. 특히, 심층 신경망을 이용하여 함수 근사를 하였고, 이때 발생하는 여러가지 장점과 수렴 해석 결과를 본 학위논문에 실었다. 마지막 문제는 상위 단계 동적 최적화 문제이다. 동적 최적화 문제에서 발생하는 제약 조건하에서 강화학습을 수행하기 위해, 원-쌍대 미분동적 계획법 (primal-dual DDP) 방법론을 새로 제안하였다. 앞서 설명한 세가지 문제에 적용된 방법론을 검증하고, 동적 계획법이 직접법에 비견될 수 있는 방법론이라는 주장을 실증하기 위해 여러가지 공정 예제를 실었다.Sequential decision making problem is a crucial technology for plant-wide process optimization. While the dominant numerical method is the forward-in-time direct optimization, it is limited to the open-loop solution and has difficulty in considering the uncertainty. Dynamic programming method complements the limitations, nonetheless associated functional optimization suffers from the curse-of-dimensionality. The sample-based approach for approximating the dynamic programming, referred to as reinforcement learning (RL) can resolve the issue and investigated throughout this thesis. The method that accounts for the system model explicitly is in particular interest. The model-based RL is exploited to solve the three representative sequential decision making problems; scheduling, supervisory optimization, and regulatory control. The problems are formulated with partially observable Markov decision process, control-affine state space model, and general state space model, and associated model-based RL algorithms are point based value iteration (PBVI), globalized dual heuristic programming (GDHP), and differential dynamic programming (DDP), respectively. The contribution for each problem can be written as follows: First, for the scheduling problem, we developed the closed-loop feedback scheme which highlights the strength compared to the direct optimization method. In the second case, the regulatory control problem is tackled by the function approximation method which relaxes the functional optimization to the finite dimensional vector space optimization. Deep neural networks (DNNs) is utilized as the approximator, and the advantages as well as the convergence analysis is performed in the thesis. Finally, for the supervisory optimization problem, we developed the novel constraint RL framework that uses the primal-dual DDP method. Various illustrative examples are demonstrated to validate the developed model-based RL algorithms and to support the thesis statement on which the dynamic programming method can be considered as a complementary method for direct optimization method.1. Introduction 1 1.1 Motivation and previous work 1 1.2 Statement of contributions 9 1.3 Outline of the thesis 11 2. Background and preliminaries 13 2.1 Optimization problem formulation and the principle of optimality 13 2.1.1 Markov decision process 15 2.1.2 State space model 19 2.2 Overview of the developed RL algorithms 28 2.2.1 Point based value iteration 28 2.2.2 Globalized dual heuristic programming 29 2.2.3 Differential dynamic programming 32 3. A POMDP framework for integrated scheduling of infrastructure maintenance and inspection 35 3.1 Introduction 35 3.2 POMDP solution algorithm 38 3.2.1 General point based value iteration 38 3.2.2 GapMin algorithm 46 3.2.3 Receding horizon POMDP 49 3.3 Problem formulation for infrastructure scheduling 54 3.3.1 State 56 3.3.2 Maintenance and inspection actions 57 3.3.3 State transition function 61 3.3.4 Cost function 67 3.3.5 Observation set and observation function 68 3.3.6 State augmentation 69 3.4 Illustrative example and simulation result 69 3.4.1 Structural point for the analysis of a high dimensional belief space 72 3.4.2 Infinite horizon policy under the natural deterioration process 72 3.4.3 Receding horizon POMDP 79 3.4.4 Validation of POMDP policy via Monte Carlo simulation 83 4. A model-based deep reinforcement learning method applied to finite-horizon optimal control of nonlinear control-affine system 88 4.1 Introduction 88 4.2 Function approximation and learning with deep neural networks 91 4.2.1 GDHP with a function approximator 91 4.2.2 Stable learning of DNNs 96 4.2.3 Overall algorithm 103 4.3 Results and discussions 107 4.3.1 Example 1: Semi-batch reactor 107 4.3.2 Example 2: Diffusion-Convection-Reaction (DCR) process 120 5. Convergence analysis of the model-based deep reinforcement learning for optimal control of nonlinear control-affine system 126 5.1 Introduction 126 5.2 Convergence proof of globalized dual heuristic programming (GDHP) 128 5.3 Function approximation with deep neural networks 137 5.3.1 Function approximation and gradient descent learning 137 5.3.2 Forward and backward propagations of DNNs 139 5.4 Convergence analysis in the deep neural networks space 141 5.4.1 Lyapunov analysis of the neural network parameter errors 141 5.4.2 Lyapunov analysis of the closed-loop stability 150 5.4.3 Overall Lyapunov function 152 5.5 Simulation results and discussions 157 5.5.1 System description 158 5.5.2 Algorithmic settings 160 5.5.3 Control result 161 6. Primal-dual differential dynamic programming for constrained dynamic optimization of continuous system 170 6.1 Introduction 170 6.2 Primal-dual differential dynamic programming for constrained dynamic optimization 172 6.2.1 Augmented Lagrangian method 172 6.2.2 Primal-dual differential dynamic programming algorithm 175 6.2.3 Overall algorithm 179 6.3 Results and discussions 179 7. Concluding remarks 186 7.1 Summary of the contributions 187 7.2 Future works 189 Bibliography 192Docto

Integration of production, maintenance and quality : Modelling and solution approaches

Dans cette thèse, nous analysons le problème de l'intégration de la planification de production et de la maintenance préventive, ainsi que l'élaboration du système de contrôle de la qualité. Premièrement, on considère un système de production composé d'une machine et de plusieurs produits dans un contexte incertain, dont les prix et le coût changent d'une période à l'autre. La machine se détériore avec le temps et sa probabilité de défaillance, ainsi que le risque de passage à un état hors contrôle augmentent. Le taux de défaillance dans un état dégradé est plus élevé et donc, des coûts liés à la qualité s’imposent. Lorsque la machine tombe en panne, une maintenance corrective ou une réparation minimale seront initiées pour la remettre en marche sans influer ses conditions ou le processus de détérioration. L'augmentation du nombre de défaillances de la machine se traduit par un temps d'arrêt supérieur et un taux de disponibilité inférieur. D'autre part, la réalisation des plans de production est fortement influencée par la disponibilité et la fiabilité de la machine. Les interactions entre la planification de la maintenance et celle de la production sont incorporées dans notre modèle mathématique. Dans la première étape, l'effet de maintenance sur la qualité est pris en compte. La maintenance préventive est considérée comme imparfaite. La condition de la machine est définie par l’âge actuel, et la machine dispose de plusieurs niveaux de maintenance avec des caractéristiques différentes (coûts, délais d'exécution et impacts sur les conditions du système). La détermination des niveaux de maintenance préventive optimaux conduit à un problème d’optimisation difficile. Un modèle de maximisation du profit est développé, dans lequel la vente des produits conformes et non conformes, les coûts de la production, les stocks tenus, la rupture de stock, la configuration de la machine, la maintenance préventive et corrective, le remplacement de la machine et le coût de la qualité sont considérés dans la fonction de l’objectif. De plus, un système composé de plusieurs machines est étudié. Dans cette extension, les nombres optimaux d’inspections est également considéré. La fonction de l’objectif consiste à minimiser le coût total qui est la somme des coûts liés à la maintenance, la production et la qualité. Ensuite, en tenant compte de la complexité des modèles préposés, nous développons des méthodes de résolution efficaces qui sont fondées sur la combinaison d'algorithmes génétiques avec des méthodes de recherches locales. On présente un algorithme mimétique qui emploi l’algorithme Nelder-Mead, avec un logiciel d'optimisation pour déterminer les valeurs exactes de plusieurs variables de décisions à chaque évaluation. La méthode de résolution proposée est comparée, en termes de temps d’exécution et de qualités des solutions, avec plusieurs méthodes Métaheuristiques. Mots-clés : Planification de la production, Maintenance préventive imparfaite, Inspection, Qualité, Modèles intégrés, MétaheuristiquesIn this thesis, we study the integrated planning of production, maintenance, and quality in multi-product, multi-period imperfect systems. First, we consider a production system composed of one machine and several products in a time-varying context. The machine deteriorates with time and so, the probability of machine failure, or the risk of a shift to an out-of-control state, increases. The defective rate in the shifted state is higher and so, quality related costs will be imposed. When the machine fails, a corrective maintenance or a minimal repair will be initiated to bring the machine in operation without influencing on its conditions or on the deterioration process. Increasing the expected number of machine failures results in a higher downtime and a lower availability rate. On the other hand, realization of the production plans is significantly influenced by the machine availability and reliability. The interactions between maintenance scheduling and production planning are incorporated in the mathematical model. In the first step, the impact of maintenance on the expected quality level is addressed. The maintenance is also imperfect and the machine conditions after maintenance can be anywhere between as-good-as-new and as-bad-as-old situations. Machine conditions are stated by its effective age, and the machine has several maintenance levels with different costs, execution times, and impacts on the system conditions. High level maintenances on the one hand have greater influences on the improvement of the system state and on the other hand, they occupy more the available production time. The optimal determination of such preventive maintenance levels to be performed at each maintenance intrusion is a challenging problem. A profit maximization model is developed, where the sale of conforming and non-conforming products, costs of production, inventory holding, backorder, setup, preventive and corrective maintenance, machine replacement, and the quality cost are addressed in the objective function. Then, a system with multiple machines is taken into account. In this extension, the number of quality inspections is involved in the joint model. The objective function minimizes the total cost which is the sum of maintenance, production and quality costs. In order to reduce the gap between the theory and the application of joint models, and taking into account the complexity of the integrated problems, we have developed an efficient solution method that is based on the combination of genetic algorithms with local search and problem specific methods. The proposed memetic algorithm employs Nelder-Mead algorithm along with an optimization package for exact determination of the values of several decision variables in each chromosome evolution. The method extracts not only the positive knowledge in good solutions, but also the negative knowledge in poor individuals to determine the algorithm transitions. The method is compared in terms of the solution time and quality to several heuristic methods. Keywords : Multi-period production planning, Imperfect preventive maintenance, Inspection, Quality, Integrated model, Metaheuristic

Classification of the Existing Knowledge Base of OR/MS Research and Practice (1990-2019) using a Proposed Classification Scheme

This is the author accepted manuscript. The final version is available from Elsevier via the DOI in this recordOperations Research/Management Science (OR/MS) has traditionally been defined as the discipline that applies advanced analytical methods to help make better and more informed decisions. The purpose of this paper is to present an analysis of the existing knowledge base of OR/MS research and practice using a proposed keywords-based approach. A conceptual structure is necessary in order to place in context the findings of our keyword analysis. Towards this we first present a classification scheme that relies on keywords that appeared in articles published in important OR/MS journals from 1990-2019 (over 82,000 articles). Our classification scheme applies a methodological approach towards keyword selection and its systematic classification, wherein approximately 1300 most frequently used keywords (in terms of cumulative percentage, these keywords and their derivations account for more than 45% of the approx. 290,000 keyword occurrences used by the authors to represent the content of their articles) were selected and organised in a classification scheme with seven top-level categories and multiple levels of sub-categories. The scheme identified the most commonly used keywords relating to OR/MS problems, modeling techniques and applications. Next, we use this proposed scheme to present an analysis of the last 30 years, in three distinct time periods, to show the changes in OR/MS literature. The contribution of the paper is thus twofold, (a) the development of a proposed discipline-based classification of keywords (like the ACM Computer Classification System and the AMS Mathematics Subject Classification), and (b) an analysis of OR/MS research and practice using the proposed classification