Towards a Methodology for the Economic Performance Increase of Production Lines using Reinforcement Learning

The increasing number of variants in product portfolios contributes to the challenge of efficient manufacturing on production lines due to the resulting small batch sizes and thus frequent product changes that lower the average overall plant effectiveness. Especially for companies that manufacture at high speed on production lines, such as in the Fast Moving Consumer Good (FMCG) industry, it is a central task of operational management to increase the performance of production lines. Due to the multitude of different adjustment levers at several interdependent machines, the identification of efficient actions and their combination into economic improvement trajectories is challenging. There is a variety of approaches to address this challenge, e.g. simulation-based heuristics. However, these approaches mostly focus on details instead of giving a holistic perspective of the possibilities to improve a production line or are limited in practical application. In other areas of application, reinforcement learning has shown remarkable success in recent years. The principle feasibility of using reinforcement learning in this application context has been demonstrated as well. However, it became apparent that the integration of expert knowledge throughout the improvement process is necessary. For this reason this paper transforms five modules defined from an engineering point of view into the mathematical scheme of a markov decision problem, a default framework for reinforcement learning. This provides the foundation for applying reinforcement learning in combination with expert knowledge from an engineering perspective

Application of a Reinforcement Learning-based Automated Order Release in Production

The importance of job shop production is increasing in order to meet the customer-driven greater demand for products with a larger number of variants in small quantities. However, it also leads to higher requirements for the production planning and control. In order to meet logistical target values and customer needs, one approach is the focus on dynamic planning systems, which can reduce ad-hoc control interventions in the running production. In particular, the release of orders at the beginning of the production process has a high influence on the planning quality. Previous approaches used advanced methods such as combinations of reinforcement learning (RL) and simulation to improve specific production environments, which are sometimes highly simplified and not practical enough. This paper presents a practice-based application of an automated order release procedure based on RL using the example of real-world production scenarios. Both, the training environment, and the data processing method are introduced. Primarily, three aspects to achieve a higher practical orientation are addressed: A more realistic problem size compared to previous approaches, a higher customer orientation by means of an objective regarding adherence to delivery date and a control application for development and performance evaluation of the considered algorithms against known order release strategies. Follow-up research will refine the objective function, continue to scale-up the problem size and evaluate the algorithm’s scheduling results in case of changes in the system