Intelligent production control for time-constrained complex job shops

Im Zuge der zunehmenden Komplexität der Produktion wird der Wunsch nach einer intelligenten Steuerung der Abläufe in der Fertigung immer größer. Sogenannte Complex Job Shops bezeichnen dabei die komplexesten Produktionsumgebungen, die deshalb ein hohes Maß an Agilität in der Steuerung erfordern. Unter diesen Umgebungen sticht die besonders Halbleiterfertigung hervor, da sie alle Komplexitäten eines Complex Job-Shop vereint. Deshalb ist die operative Exzellenz der Schlüssel zum Erfolg in der Halbleiterindustrie. Diese Exzellenz hängt ganz entscheidend von einer intelligenten Produktionssteuerung ab. Ein Hauptproblem bei der Steuerung solcher Complex Job-Shops, in diesem Fall der Halbleiterfertigung, ist das Vorhandensein von Zeitbeschränkungen (sog. time-constraints), die die Transitionszeit von Produkten zwischen zwei, meist aufeinanderfolgenden, Prozessen begrenzen. Die Einhaltung dieser produktspezifischen Zeitvorgaben ist von größter Bedeutung, da Verstöße zum Verlust des betreffenden Produkts führen. Der Stand der Technik bei der Produktionssteuerung dieser Dispositionsentscheidungen, die auf die Einhaltung der Zeitvorgaben abzielen, basiert auf einer fehleranfälligen und für die Mitarbeiter belastenden manuellen Steuerung. In dieser Arbeit wird daher ein neuartiger, echtzeitdatenbasierter Ansatz zur intelligenten Steuerung der Produktionssteuerung für time-constrained Complex Job Shops vorgestellt. Unter Verwendung einer jederzeit aktuellen Replikation des realen Systems werden sowohl je ein uni-, multivariates Zeitreihenmodell als auch ein digitaler Zwilling genutzt, um Vorhersagen über die Verletzung dieser time-constraints zu erhalten. In einem zweiten Schritt wird auf der Grundlage der Erwartung von Zeitüberschreitungen die Produktionssteuerung abgeleitet und mit Echtzeitdaten anhand eines realen Halbleiterwerks implementiert. Der daraus resultierende Ansatz wird gemeinsam mit dem Stand der Technik validiert und zeigt signifikante Verbesserungen, da viele Verletzungen von time-constraints verhindert werden können. Zukünftig soll die intelligente Produktionssteuerung daher in weiteren Complex Job Shop-Umgebungen evaluiert und ausgerollt werden

Reinforcement learning for energy-efficient control of multi-stage production lines with parallel machine workstations

An effective approach to enhancing the sustainability of production systems is to use energy-efficient control (EEC) policies for optimal balancing of production rate and energy demand. Reinforcement learning (RL) algorithms can be employed to successfully control production systems, even when there is a lack of prior knowledge about system parameters. Furthermore, recent research demonstrated that RL can be also applied for the optimal EEC of a single manufacturing workstation with parallel machines. The purpose of this study is to apply an RL for EEC approach to more workstations belonging to the same industrial production system from the automotive sector, without relying on full knowledge of system dynamics. This work aims to show how the RL for EEC of more workstations affects the overall production system in terms of throughput and energy consumption. Numerical results demonstrate the benefits of the proposed model

Framework for automatic production simulation tuning with machine learning

Production system simulation is a powerful tool for optimizing the use of resources on both the planning and control level. However, creating and tuning such models manually is a tedious and error-prone task. Despite some approaches to automate this process, the state-of-the-art relies on the generation of models, by incorporating the knowledge of experts. Nevertheless, effectively creating and tuning such production simulations is, thus, a key driver for reducing costs, carbon footprint, and tardiness and therefore an essential factor in today´s production. Beneficial would be automated and flexible frameworks, since these are applicable to different use cases requiring less effort. Yet, in the age of Industry 4.0, data is ubiquitous and easily available and can serve as a basis for virtual models representing reality. Increasingly, these virtual models shall be interlinked with the current state of real-world systems to form so-called digital twins. As automated and flexible frameworks are missing, this paper proposes a novel approach where observed real system behavior is used and fed into a large-scale machine learning model trained on a plethora of possible parameter sets. The main target is to train this machine learning model to minimize the reality gap between the behavior of the simulated and real system by selecting corresponding simulation system parameters. By estimating those parameters an enhancement of the simulation will emerge. An interlink to real systems can be derived resulting in a digital shadow which is capable to forecast the future similarly to reality. The approach to overcoming the gap between reality and simulation (real2sim) is validated in simulations

Automated Derivation of Optimal Production Sequences from Product Data

Customer specific, individual products nowadays lead to larger product variance and shorter time to market. This requires efficient production system planning. In addition, due to a larger system complexity, each iteration of the planning process itself gets soaringly complex. Time constraints and complexity, therefore, emphasize the necessity of supporting humans in planning modern production systems. Especially the determination of the production sequence holds immense potential and tends to get even more complex within specific production technologies. Exemplarily, this article focuses on welding sequences. Here, domain knowledge from product development and production planning needs to be holistically integrated. Furthermore, implicit, historic knowledge needs to be formalized and used in today’s planning tasks. This article introduces a methodical approach and a corresponding toolchain to derive optimal production sequences from customer product data which is validated using welding processes. For this, firstly, a reference system is build up consisting of historic product data (e.g. part list, CAD data) and corresponding production system characteristics (e.g. number and specifications of machines). The main aspect is to use similarities between the new product variant and assemblies from the reference system, to determine implications of product specifications on the process sequence. Overall, such restrictions can be displayed using Model-Based Systems Engineering. Relevant information (e.g. weld seam lengths) can be used to compute the optimal weld seam order regarding minimal cycle times, for example. This requires a parametric encoding of product and production system. In a nutshell, this approach covers the automated derivation of an optimal production sequence for new product variants, based on system information and product similarities, to tackle time constraints and complexity by suggesting initial planning drafts

Reinforcement Learning Based Production Control of Semi-automated Manufacturing Systems

In an environment which is marked by an increasing speed of changes, industrial companies have to be able to quickly adapt to new market demands and innovative technologies. This leads to a need for continuous adaption of existing production systems and the optimization of their production control. To tackle this problem digitalization of production systems has become essential for new and existing systems. Digital twins based on simulations of real production systems allow the simplification of analysis processes and, thus, a better understanding of the systems, which leads to broad optimization possibilities. In parallel, machine learning methods can be integrated to process the numerical data and discover new production control strategies. In this work, these two methods are combined to derive a production control logic in a semi-automated production system based on the chaku-chaku principle. A reinforcement learning method is integrated into the digital twin to autonomously learn a superior production control logic for the distribution of tasks between the different workers on a production line. By analyzing the influence of different reward shaping and hyper-parameter optimization on the quality and stability of the results obtained, the use of a well-configured policy-based algorithm enables an efficient management of the workers and the deduction of an optimal production control logic for the production system. The algorithm manages to define a control logic that leads to an increase in productivity while having a stable task assignment so that a transfer to daily business is possible. The approach is validated in the digital twin of a real assembly line of an automotive supplier. The results obtained suggest a new approach to optimizing production control in production lines. Production control shall be centered directly on the workers’ routines and controlled by artificial intelligence infused with a global overview of the entire production system

Decentralized Multi-Agent Production Control through Economic Model Bidding for Matrix Production Systems

Due to increasing demand for unique products, large variety in product portfolios and the associated rise in individualization, the efficient use of resources in traditional line production dwindles. One answer to these new challenges is the application of matrix-shaped layouts with multiple production cells, called Matrix Production Systems. The cycle time independence and redundancy of production cell capabilities within a Matrix Production System enable individual production paths per job for Flexible Mass Customisation. However, the increased degrees of freedom strengthen the need for reliable production control systems compared to traditional production systems such as line production. Beyond reliability a need for intelligent production within a smart factory in order to ensure goal-oriented production control under ever-changing manufacturing conditions can be ascertained. Learning-based methods can leverage condition-based reactions for goal-oriented production control. While centralized control performs well in single-objective situations, it is hard to achieve contradictory targets for individual products or resources. Hence, in order to master these challenges, a production control concept based on a decentralized multi-agent bidding system is presented. In this price-based model, individual production agents - jobs, production cells and transport system - interact based on an economic model and attempt to maximize monetary revenues. Evaluating the application of learning and priority-based control policies shows that decentralized multi-agent production control can outperform traditional approaches for certain control objectives. The introduction of decentralized multi-agent reinforcement learning systems is a starting point for further research in this area of intelligent production control within smart manufacturing