Application of Reinforcement Learning to Multi-Agent Production Scheduling

Reinforcement learning (RL) has received attention in recent years from agent-based researchers because it can be applied to problems where autonomous agents learn to select proper actions for achieving their goals based on interactions with their environment. Each time an agent performs an action, the environment¡Šs response, as indicated by its new state, is used by the agent to reward or penalize its action. The agent¡Šs goal is to maximize the total amount of reward it receives over the long run. Although there have been several successful examples demonstrating the usefulness of RL, its application to manufacturing systems has not been fully explored. The objective of this research is to develop a set of guidelines for applying the Q-learning algorithm to enable an individual agent to develop a decision making policy for use in agent-based production scheduling applications such as dispatching rule selection and job routing. For the dispatching rule selection problem, a single machine agent employs the Q-learning algorithm to develop a decision-making policy on selecting the appropriate dispatching rule from among three given dispatching rules. In the job routing problem, a simulated job shop system is used for examining the implementation of the Q-learning algorithm for use by job agents when making routing decisions in such an environment. Two factorial experiment designs for studying the settings used to apply Q-learning to the single machine dispatching rule selection problem and the job routing problem are carried out. This study not only investigates the main effects of this Q-learning application but also provides recommendations for factor settings and useful guidelines for future applications of Q-learning to agent-based production scheduling

A random search approach to the machine loading problem of an FMS

This paper discusses a modelling framework that addresses operational planning, problems of flexible manufacturing systems (FMSs). A generic 0-1 mixed integer programming formulation integrating the part selection and loading problems has been proposed. The constraints considered in the problems are mainly the availability of tool slots and machining time on the machining centres. The above problem is solved using an algorithm based on Simulated Annealing (SA). The potential capability of the approach is demonstrated via a small set of test problems. ©2004 IEEE.published_or_final_versio

An Integrated Approach for the Analysis of Manufacturing System States

With advancement in the manufacturing technology and rise in the purchasing ability, demand for newer products is increasing continuously. This is forcing manufacturing companies to persistently look for new techniques to improve the productivity of a manufacturing system and ensure optimum utilization of all the elements of a manufacturing system, including facility layout. Traditional research had viewed facility layout, material handling and productivity improvement as separate activities.  Researchers depending on their area of specialization focused on either the production aspects of a company, the material handling aspects or facility layout. However, to ensure productivity, this study proposes a new theory to analyze the current state of the system with an integrated approach of production system and material handling system. In this study, the current state of the system is classified into three different states and a methodology is proposed to identify the current state of the system. This new theory can be used by manufacturers to identify appropriate strategies for improving productivity.  The identification of the state of the system is necessary for effective improvement of the system

An investigation into tooling requirements and strategies for FMS operation

A study of the minimum tooling requirements and strategies for efficient operation of Flexible Manufacturing Systems, FMS's, in Assembly set Production, ASP, i.e production in sets of parts to completely assemble one or more product units, is presented in this research work. The main investigating tool is a simulation model. With this model the tool groups to be loaded into machines and fixtured pallet requirements were studied in conjunction with two scheduling rules. One is a FCFS rule and the other is a new rule, called MRPAS, which schedules work on the basis of the number of parts still unfinished belonging to an Assembly Set. The results of the research work show that ASP can be efficiently carried out in FMS's. However this requires that a good system set-up and adequate operating strategies are used. In particular appropriate tooling levels and good tooling configurations,TC's, i.e. combinations of tools in groups to be loaded into the machines, must be established to achieve high FMS performance. Tooling combination and duplication heuristic rules and the simulation model can be used for achieving this aim. The heuristic approach is shown to be necessary due to the impossibility, in a reasonable time, of evaluating the performance of FMS's under the large number of alternative tooling configurations which are possible. The level of fixtured pallets used can also have a great influence on system performance. Appropriate levels of these resources to operate FMS's for given TC's can be established using the methodology developed in this work. It is also important that good scheduling rules are used. In the cases studied, the MRPAS rule produces the best performance expressed as the combination of FMS utilization and production of complete assembly sets. Moreover a very small assembly set batch size, ASBS, i.e. number of AS released together into the FMS, is likely to be preferable. In the cases studied an ASBS of one performed best overall

The investigation of the effect of scheduling rules on FMS performance

The application of Flexible Manufacturing Systems (FMSs) has an effect in competitiveness, not only of individual companies but of those countries whose manufactured exports play a significant part in their economy (Hartley, 1984). However, the increasing use of FM Ss to effectively provide customers with diversified products has created a significant set of operational challenges for managers (Mahmoodi et al., 1999). In more recent years therefore, there has been a concentration of effort on FMS scheduling without which the benefits of an FMS cannot be realized. The objective of the reported research is to investigate and extend the contribution which can be made to the FMS scheduling problem through the implementation of computer-based experiments that consider real-time situations. [Continues.

Machine Loading in Flexible Manufacturing system using Artificial Immune algorithm

In this project thesis the FMS loading problem is discussed with the objective to minimize the system unbalance and throughput by the use of Artificial Immune system. Manufacturing technology focuses primarily on flexibility and productivity. With the product variety and product life being the characterizing standards it is important that the flexibility of the job shop is maintained as its efficiency is increases. The complexity of a basic Machine loading problem in FMS is very high due to the different flexibility criteria as Part selection, Operation allocation and the various constraints involved. This dissertation proposes a soft computing technique with constraints on tool capacity and workload of the machine. The aim of using this algorithm is to reach an optimal solution and to ease the tedious computations in large problems involving loading which are NP hard problems. Immune algorithm is a very suitable method due to its self learning and memory acquisition abilities. First some sample machine loading problems are collected from the literature and the optimal system unbalance of the machine is calculated using LINGO optimization software. This project improves some issues inherent in existing techniques and proposes an effective Immune algorithm with reduced memory requirements and reduced computational complexity. The proposed Algorithm is tested on 3 problems adopted from literatures and the results reveal substantial improvement in solution quality over the existing basic mathematical approaches

Intelligent shop scheduling for semiconductor manufacturing

Semiconductor market sales have expanded massively to more than 200 billion dollars annually accompanied by increased pressure on the manufacturers to provide higher quality products at lower cost to remain competitive. Scheduling of semiconductor manufacturing is one of the keys to increasing productivity, however the complexity of manufacturing high capacity semiconductor devices and the cost considerations mean that it is impossible to experiment within the facility. There is an immense need for effective decision support models, characterizing and analyzing the manufacturing process, allowing the effect of changes in the production environment to be predicted in order to increase utilization and enhance system performance. Although many simulation models have been developed within semiconductor manufacturing very little research on the simulation of the photolithography process has been reported even though semiconductor manufacturers have recognized that the scheduling of photolithography is one of the most important and challenging tasks due to complex nature of the process. Traditional scheduling techniques and existing approaches show some benefits for solving small and medium sized, straightforward scheduling problems. However, they have had limited success in solving complex scheduling problems with stochastic elements in an economic timeframe. This thesis presents a new methodology combining advanced solution approaches such as simulation, artificial intelligence, system modeling and Taguchi methods, to schedule a photolithography toolset. A new structured approach was developed to effectively support building the simulation models. A single tool and complete toolset model were developed using this approach and shown to have less than 4% deviation from actual production values. The use of an intelligent scheduling agent for the toolset model shows an average of 15% improvement in simulated throughput time and is currently in use for scheduling the photolithography toolset in a manufacturing plant

Intelligent Simulation Modeling of a Flexible Manufacturing System with Automated Guided Vehicles

Although simulation is a very flexible and cost effective problem solving technique, it has been traditionally limited to building models which are merely descriptive of the system under study. Relatively new approaches combine improvement heuristics and artificial intelligence with simulation to provide prescriptive power in simulation modeling. This study demonstrates the synergy obtained by bringing together the "learning automata theory" and simulation analysis. Intelligent objects are embedded in the simulation model of a Flexible Manufacturing System (FMS), in which Automated Guided Vehicles (AGVs) serve as the material handling system between four unique workcenters. The objective of the study is to find satisfactory AGV routing patterns along available paths to minimize the mean time spent by different kinds of parts in the system. System parameters such as different part routing and processing time requirements, arrivals distribution, number of palettes, available paths between workcenters, number and speed of AGVs can be defined by the user. The network of learning automata acts as the decision maker driving the simulation, and the FMS model acts as the training environment for the automata network; providing realistic, yet cost-effective and risk-free feedback. Object oriented design and implementation of the simulation model with a process oriented world view, graphical animation and visually interactive simulation (using GUI objects such as windows, menus, dialog boxes; mouse sensitive dynamic automaton trace charts and dynamic graphical statistical monitoring) are other issues dealt with in the study

A New Multicommodity Flow Model for the Job Sequencing and Tool Switching Problem

Artigo científico.In this paper a new multicommodity flow mathematical model for the Job Sequencing and Tool Switching Problem (SSP) is presented. The proposed model has a LP relaxation lower bound equal to the number of tools minus the tool machine’s capacity. Computational tests were performed comparing the new model with the models of the literature. The proposed model performed better, both in execution time and in the number of instances solved to optimality.Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES