Reinforcement learning for an intelligent and autonomous production control of complex job-shops under time constraints

Reinforcement learning (RL) offers promising opportunities to handle the ever-increasing complexity in managing modern production systems. We apply a Q-learning algorithm in combination with a process-based discrete-event simulation in order to train a self-learning, intelligent, and autonomous agent for the decision problem of order dispatching in a complex job shop with strict time constraints. For the first time, we combine RL in production control with strict time constraints. The simulation represents the characteristics of complex job shops typically found in semiconductor manufacturing. A real-world use case from a wafer fab is addressed with a developed and implemented framework. The performance of an RL approach and benchmark heuristics are compared. It is shown that RL can be successfully applied to manage order dispatching in a complex environment including time constraints. An RL-agent with a gain function rewarding the selection of the least critical order with respect to time-constraints beats heuristic rules strictly by picking the most critical lot first. Hence, this work demonstrates that a self-learning agent can successfully manage time constraints with the agent performing better than the traditional benchmark, a time-constraint heuristic combining due date deviations and a classical first-in-first-out approach

Multi-variate time-series for time constraint adherence prediction in complex job shops

One of the most complex and agile production environments is semiconductor manufacturing, especially wafer fabrication, as products require more than several hundred operations and remain in Work-In-Progress for months leading to complex job shops. Additionally, an increasingly competitive market environment, i.e. owing to Moore’s law, forces semiconductor companies to focus on operational excellence, resiliency and, hence, leads to product quality as a decisive factor. Product-specific time constraints comprising two or more, not necessarily consecutive, operations ensure product quality at an operational level and, thus, are an industry-specific challenge. Time constraint adherence is of utmost importance, since violations typically lead to scrapping entire lots and a deteriorating yield. Dispatching decisions that determine time constraint adherence are as a state of the art performed manually, which is stressful and error-prone. Therefore, this article presents a data-driven approach combining multi-variate time-series with centralized information to predict time constraint adherence probability in wafer fabrication to facilitate dispatching. Real-world data is analyzed and different statistical and machine learning models are evaluated

Deep reinforcement learning für workload balance und Fälligkeitskontrolle in wafer fabs

Semiconductor wafer fabrication facilities (wafer fabs) often prioritize two operational objectives: work-in-process (WIP) and due date. WIP-oriented and due date-oriented dispatching rules are two commonly used methods to achieve workload balance and on-time delivery, respectively. However, it often requires sophisticated heuristics to achieve both objectives simultaneously. In this paper, we propose a novel approach using deep-Q-network reinforcement learning (DRL) for dispatching in wafer fabs. The DRL approach differs from traditional dispatching methods by using dispatch agents at work-centers to observe state changes in the wafer fabs. The agents train their deep-Q-networks by taking the states as inputs, allowing them to select the most appropriate dispatch action. Additionally, the reward function is integrated with workload and due date information on both local and global levels. Compared to the traditional WIP and due date-oriented rules, as well as heuristics-based rule in literature, the DRL approach is able to produce better global performance with regard to workload balance and on-time delivery

Job Flows and Establishment Characteristics: Variations Across U.S. Metropolitan Areas

This paper addresses the role played within metropolitan areas by heterogeneous agent models of constant churning. The evidence shows positive relationships between job turnover, young establishments, and metropolitan employment growth. Most areas, however, differ in their levels of job creation rather than job destruction. Results persist after controlling for regional differences in industry, but less so when controlling for differences in the establishment age distribution, and are consistent overall with standard models of creative destruction. Evidence from several entering cohorts, however, contradicts the vintage replacement process of creative destruction models. Namely, job destruction decreases as establishments age and there is no clear inverse relation between establishment entry rates and exit ages. These patterns are instead consistent with a turnover process seen in standard models of firm learning. Further evidence suggests that these patterns vary systematically with the overall employment growth of a region. Together, the results suggest that (i) processes of both creative destruction and firm learning may matter for local labor dynamics, but future models will have to reconcile with this new evidence, and (ii) intrinsic local factors, such as the “business climate”, may affect the dynamics of both processes.http://deepblue.lib.umich.edu/bitstream/2027.42/39995/3/wp609.pd

Scheduling Algorithms: Challenges Towards Smart Manufacturing

Collecting, processing, analyzing, and driving knowledge from large-scale real-time data is now realized with the emergence of Artificial Intelligence (AI) and Deep Learning (DL). The breakthrough of Industry 4.0 lays a foundation for intelligent manufacturing. However, implementation challenges of scheduling algorithms in the context of smart manufacturing are not yet comprehensively studied. The purpose of this study is to show the scheduling No.s that need to be considered in the smart manufacturing paradigm. To attain this objective, the literature review is conducted in five stages using publish or perish tools from different sources such as Scopus, Pubmed, Crossref, and Google Scholar. As a result, the first contribution of this study is a critical analysis of existing production scheduling algorithms\u27 characteristics and limitations from the viewpoint of smart manufacturing. The other contribution is to suggest the best strategies for selecting scheduling algorithms in a real-world scenario

Job Flows and Establishment Characteristics: Variations Across U.S. Metropolitan Areas

This paper addresses the role played within metropolitan areas by heterogeneous agent models of constant churning. The evidence shows positive relationships between job turnover, young establishments, and metropolitan employment growth. Most areas, however, differ in their levels of job creation rather than job destruction. Results persist after controlling for regional differences in industry, but less so when controlling for differences in the establishment age distribution, and are consistent overall with standard models of creative destruction. Evidence from several entering cohorts, however, contradicts the vintage replacement process of creative destruction models. Namely, job destruction decreases as establishments age and there is no clear inverse relation between establishment entry rates and exit ages. These patterns are instead consistent with a turnover process seen in standard models of firm learning. Further evidence suggests that these patterns vary systematically with the overall employment growth of a region. Together, the results suggest that (i) processes of both creative destruction and firm learning may matter for local labor dynamics, but future models will have to reconcile with this new evidence, and (ii) intrinsic local factors, such as the “business climate”, may affect the dynamics of both processes.job turnover, regional and urban growth, creative destruction, firm learning

Hybrid ASP-based multi-objective scheduling of semiconductor manufacturing processes (Extended version)

Modern semiconductor manufacturing involves intricate production processes consisting of hundreds of operations, which can take several months from lot release to completion. The high-tech machines used in these processes are diverse, operate on individual wafers, lots, or batches in multiple stages, and necessitate product-specific setups and specialized maintenance procedures. This situation is different from traditional job-shop scheduling scenarios, which have less complex production processes and machines, and mainly focus on solving highly combinatorial but abstract scheduling problems. In this work, we address the scheduling of realistic semiconductor manufacturing processes by modeling their specific requirements using hybrid Answer Set Programming with difference logic, incorporating flexible machine processing, setup, batching and maintenance operations. Unlike existing methods that schedule semiconductor manufacturing processes locally with greedy heuristics or by independently optimizing specific machine group allocations, we examine the potentials of large-scale scheduling subject to multiple optimization objectives.Comment: 17 pages, 1 figure, 4 listings, 1 table; a short version of this paper is presented at the 18th European Conference on Logics in Artificial Intelligence (JELIA 2023

반도체 공장 내 일시적인 생산 용량 확장 정책 제안

학위논문 (석사) -- 서울대학교 대학원 : 공과대학 산업공학과, 2021. 2. 박건수.Due to the instability of the capacity of the semiconductor process, there are cases in which the production capacity temporarily becomes insufficient compared to the capacity allocated by the initial plan. To respond, production managers require capacity to other lines with compatible equipment. This decision can have an adverse effect on the entire line because the processes are connected in a sequence. In particular, it becomes more problematic when the machine group is a bottleneck process group. Therefore, this study proposes a capacity expansion policy learned by reinforcement learning algorithms in this environment using a FAB simulator built upon a WIP balancing scheduler and a machine disruption model. These policies performed better than policies imitating human decision in terms of throughput and machine efficiency.반도체공장은 설비 용량의 불안정성 때문에 초기 계획하여 할당된 설비 용량에 비해 일시적으로 생산 용량이 부족해지는 경우가 발생한다. 이를 대응하기 위해 생산 담당자들은 다른 라인에 호환가능한 설비를 공유하는 것을 요청하는데, 가능한 많은 양의 WIP에 대한 요청을 한다. 이러한 의사결정은 공정이 순차적으로 연결된 점 때문에 라인 전체 측면에서는 오히려 WIP Balancing을 악화시킬 수 있다. 특히 해당 공정군이 병목공정군인 경우 더 문제가 된다. 따라서 본 연구에서는 병목공정군을 중심으로 한 WIP Balancing scheduler를 이용하여 FAB simulator를 만든 뒤 이러한 환경속에서 강화학습 알고리즘으로 학습한 생산 용량 확장 정책을 제안한다. 이러한 정책은 throughput, machine efficiency 측면에서 사람의 의사결정을 모방한 정책보다 좋은 성과를 보였다.Abstract i Contents ii List of Tables iv List of Figures v Chapter 1 Introduction 1 1.1 Problem Description 3 1.2 Research Motivation and Contribution 5 1.3 Organization of the Thesis 5 Chapter 2 Literature Review 6 2.1 Review on FAB scheduling 6 2.2 Review on Dynamic production control 7 Chapter 3 Proposed Approach and Methodology 8 3.1 Proposed Approach 8 3.2 FAB Simulator 17 3.3 Reinforcement Learning Approach 26 Chapter 4 Computational Experiments 30 4.1 Experiment settings 30 4.2 Test Instances 31 4.3 Test Results 33 Chapter 5 Conclusions 37 Bibliography 38 국문초록 39Maste

Auxetic materials and structures for potential defense applications: an overview and recent developments

Auxetic behavior is a promising new area for use in defense applications. In comparison to a conventional material, an auxetic material has superior properties because of having a negative Poisson’s ratio; it gets broadened when stretched or becomes smaller when compressed. Furthermore, auxetic materials have enhanced properties such as shear resistance, indentation resistance, fracture toughness, energy absorption, and so on. These improved properties make auxetic materials very appealing and have the potential to revolutionize their applications in aerospace, sports, auto motive, construction, biomedical engineering, smart sensors, packaging, cushioning, air filtration, shock absorption and sound insulation, and defense personal protective equipment. This article examines the most recent scientific advances in auxetic materials and structures, such as auxetic textiles (fibers, yarns, and fabrics), auxetic textile-reinforced composites, and auxetic foams, as well as their exceptional auxetic behavior and various approaches to achieving them. Although many potential applications have been proposed, actual applications of auxetic materials in defense are still limited. This is an in-depth review article, and its main goal is to serve as a foundation for future studies concerning the topic.The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The authors acknowledge for the financial support from the Portuguese Foundation for Science and Technology (FCT) through project UID/CTM/00264/2019 of 2C2T - Centro de Cieˆncia e Tecnologia Teˆxtil, by national funds of FCT/MCTES