Design, Implementation and Evaluation of Reinforcement Learning for an Adaptive Order Dispatching in Job Shop Manufacturing Systems

Modern production systems tend to have smaller batch sizes, a larger product variety and more complex material flow systems. Since a human oftentimes can no longer act in a sufficient manner as a decision maker under these circumstances, the demand for efficient and adaptive control systems is rising. This paper introduces a methodical approach as well as guideline for the design, implementation and evaluation of Reinforcement Learning (RL) algorithms for an adaptive order dispatching. Thereby, it addresses production engineers willing to apply RL. Moreover, a real-world use case shows the successful application of the method and remarkable results supporting real-time decision-making. These findings comprehensively illustrate and extend the knowledge on RL

Data Analytics for Manufacturing Systems – A Data-Driven Approach for Process Optimization

In the course of digitalization many small and medium-sized companies face the challenge of using the existing database for process optimization in manufacturing. Furthermore, the demand-oriented expansion of the database is a great challenge. A lack of competencies, limited financial resources and historically grown data structures, which show a strong heterogeneity and lack of transparency, are the central obstacles. A specific approach, how data analytics projects for process optimization should be carried out in manufacturing, is presented. In particular, the question which sensors should be implemented to expand the database is answered. The approach is applied exemplarily for a manufacturing line

Variant flexibility in assembly line balancing under the premise of feasibility robustness

The final assembly of vehicles is frequently designed as a mixed model line which produces effectively at a fixed ratio of variants. Market forecasts indicate a volatile future demand for different types of vehicles such as electrified vehicles. The resulting uncertainty of demand affects the task of line balancing assembly lines. This paper presents a new planning method to provide decision support for line balancing with inherent variant flexibility while maintaining feasibility robustness. Therefore a combined approach of scenario analyses and line balancing optimization is developed. This approach is applied to a use case in automotive industry

Deep Learning for Automated Product Design

Product development is a highly complex process that has to be individually adapted depending on the companies involved, the product to be developed and the related designers. Within this process, the approach and the know-how of the designer are very individual and can often only be described with high effort in a rule-based manner. Nevertheless, numerous routine tasks can be identified that offer enormous automation potential. Machine Learning, especially Deep Learning, has proven an immense capability to identify patterns and extract knowledge out of complex data sets. Autoencoder networks are suitable for the conversion of different 3D input data, e.g. Point Clouds, into compact latent representations and vice versa. Point Clouds are a universal representation of 3D objects and can be derived from various 3D data formats. The goal of the approach presented is to use Deep Learning algorithms to identify design patterns specific to a product family out of their underlying latent representation and use the extracted knowledge to automatically generate new latent object representations fulfilling distinct product feature specifications. A deep Autoencoder network with state-of-the-art reconstruction quality is used to encode Point Clouds into latent representations. In this approach, a conditional Generative Adversarial Network operating in latent space for generation of class-, characteristic- and dimension-conditioned objects is introduced. The model is quantitatively evaluated by a comparison of given specifications and the implemented features of generated objects. The presented findings can be used to support designers in the creation process by automatically proposing appropriate objects as well as in the adaption of future product variants to different requirements. This relieves the designer of time-consuming routine tasks and reduces the effort of knowledge-transfer between designers significantly

Fluid Automation - A Definition and an Application in Remanufacturing Production Systems

Production systems must be able to quickly adapt to changing requirements. Especially in the field of remanufacturing, the uncertainty in the state of the incoming products is very high. Several adaptation mechanisms can be applied leading to agile and changeable production systems. Among these, adapting the degree of automation with respect to changeover times and high investment costs is one of the most challenging mechanisms. However, not only long-term changes, but also short-term adaptations can lead to enormous potentials, e.g. when night shifts can be supported by robots and thus higher labor costs and unfavorable working conditions at night can be avoided. These changes in the degree of automation on an operational level are referred to as fluid automation, which will be defined in this paper. The mechanisms of fluid automation are presented together with a case study showing its application on a disassembly station for electrical drives

Integrating product function design, production technology optimization and process equipment planning on the example of hybrid additive manufacturing

New technologies can yield high market potential, but also challenge engineering capabilities. For example, additive manufacturing enables unlimited freedom of design and economical production of small batch sizes. However, there are huge challenges: A large variety of new additive technologies, limited choice of materials and mostly high production cost as result of long production time. Since today’s production requires an economical implementation, focus needs to be on hybrid production, which combines the advantages of additive and conventional manufacturing technologies. This requires an integrated optimization of the product design, the manufacturing technology chain and the operative equipment. The following paper presents an approach for this integrated planning approach with the aim of economically feasible hybrid production. In general, the interdependencies between product and manufacturing technology need to be used for optimization in early stages of the product life cycle. To achieve a high customer value, the product requirements have to be analyzed in detail to find an optimal product function, but also to identify degrees of design freedom, which do not influence product function and, thus, can be adapted to optimize production. Moreover, possible changes in the capabilities of manufacturing technologies and, subsequently, operative equipment and machines can be anticipated to further enhance the production. After identifying optimal combinations of product design and manufacturing technology chains, the selection and optimal configuration of the operative equipment is necessary and needs to be validate based on the final product design. The integration of product design, manufacturing technology optimization and operative process planning enables companies to identify and realize high economic potential early in their value creation process and thus can contribute to improving competitiveness

Decentralized Multi-Agent Production Control through Economic Model Bidding for Matrix Production Systems

Due to increasing demand for unique products, large variety in product portfolios and the associated rise in individualization, the efficient use of resources in traditional line production dwindles. One answer to these new challenges is the application of matrix-shaped layouts with multiple production cells, called Matrix Production Systems. The cycle time independence and redundancy of production cell capabilities within a Matrix Production System enable individual production paths per job for Flexible Mass Customisation. However, the increased degrees of freedom strengthen the need for reliable production control systems compared to traditional production systems such as line production. Beyond reliability a need for intelligent production within a smart factory in order to ensure goal-oriented production control under ever-changing manufacturing conditions can be ascertained. Learning-based methods can leverage condition-based reactions for goal-oriented production control. While centralized control performs well in single-objective situations, it is hard to achieve contradictory targets for individual products or resources. Hence, in order to master these challenges, a production control concept based on a decentralized multi-agent bidding system is presented. In this price-based model, individual production agents - jobs, production cells and transport system - interact based on an economic model and attempt to maximize monetary revenues. Evaluating the application of learning and priority-based control policies shows that decentralized multi-agent production control can outperform traditional approaches for certain control objectives. The introduction of decentralized multi-agent reinforcement learning systems is a starting point for further research in this area of intelligent production control within smart manufacturing

Towards planning and control in cognitive factories - A generic model including learning effects and knowledge transfer across system entities

Cognitive abilities allow robots to learn and reason from their environment. The gained knowledge can then be incorporated into the robot’s actions which in turn affect the environment. Therefore, a cognitive robot is no longer a static system that performs actions based on a pre-defined set of rules but a complex entity that dynamically adjusts over time. With this, challenges arise for production systems that need to observe and ideally anticipate the cognitive robot’s behavior. Often, digital twins are employed to test and optimize production control systems. This paper presents a generic approach to characterize, model and simulate learning processes and formalized knowledge in hybrid production systems assuming different station types with learning effects. Thereby, quantitative and qualitative learning processes are mapped including knowledge sharing and transfer across entities. A modular and parameterizable design enables the adjustment to different use cases. Eventually, the model is instantiated as a digital twin of a real production system for product disassembly employing cognitive-autonomous robots among human operators and rigidly automated machines. The model shows great potential to be integrated into test beds for planning and control systems of cognitive factories