Solving the Order Batching and Sequencing Problem using Deep Reinforcement Learning

In e-commerce markets, on time delivery is of great importance to customer satisfaction. In this paper, we present a Deep Reinforcement Learning (DRL) approach for deciding how and when orders should be batched and picked in a warehouse to minimize the number of tardy orders. In particular, the technique facilitates making decisions on whether an order should be picked individually (pick-by-order) or picked in a batch with other orders (pick-by-batch), and if so with which other orders. We approach the problem by formulating it as a semi-Markov decision process and develop a vector-based state representation that includes the characteristics of the warehouse system. This allows us to create a deep reinforcement learning solution that learns a strategy by interacting with the environment and solve the problem with a proximal policy optimization algorithm. We evaluate the performance of the proposed DRL approach by comparing it with several batching and sequencing heuristics in different problem settings. The results show that the DRL approach is able to develop a strategy that produces consistent, good solutions and performs better than the proposed heuristics.Comment: Preprin

Design and Control of Warehouse Order Picking: a literature review

Order picking has long been identified as the most labour-intensive and costly activity for almost every warehouse; the cost of order picking is estimated to be as much as 55% of the total warehouse operating expense. Any underperformance in order picking can lead to unsatisfactory service and high operational cost for its warehouse, and consequently for the whole supply chain. In order to operate efficiently, the orderpicking process needs to be robustly designed and optimally controlled. This paper gives a literature overview on typical decision problems in design and control of manual order-picking processes. We focus on optimal (internal) layout design, storage assignment methods, routing methods, order batching and zoning. The research in this area has grown rapidly recently. Still, combinations of the above areas have hardly been explored. Order-picking system developments in practice lead to promising new research directions.Order picking;Logistics;Warehouse Management

Integrated Batching and Lot Streaming with Variable Sublots and Sequence-Dependent Setups in a Two-Stage Hybrid Flow Shop

Consider a paint manufacturing firm whose customers typically place orders for two or more products simultaneously: liquid primer, top coat paint, and/or undercoat paint. Each product belongs to an associated product family that can be batched together during the manufacturing process. Meanwhile, each product can be split into several sublots so that overlapping production is possible in a two-stage hybrid flow shop. Various numbers of identical capacitated machines operate in parallel at each stage. We present a mixed-integer programming (MIP) to analyze this novel integrated batching and lot streaming problem with variable sublots, incompatible job families, and sequence-dependent setup times. The model determines the number of sublots for each product, the size of each sublot, and the production sequencing for each sublot such that the sum of weighted completion time is minimized. Several numerical example problems are presented to validate the proposed formulation and to compare results with similar problems in the literature. Furthermore, an experimental design based on real industrial data is used to evaluate the performance of proposed model. Results indicate that the computational cost of solving the model is high

Variable Neighborhood Search for the Order Batching and Sequencing Problem with Multiple Pickers

Order picking deals with the retrieval of articles from their storage locations in order to satisfy customer requests. The transformation and consolidation of customer orders into picking orders (batches) is pivotal for the performance of order picking systems. Typically, customer orders have to be completed by certain due dates in order to avoid delays in production or in the shipment to customers. The composition of the batches, their processing times, their assignment to order pickers and the sequence according to which they are scheduled determine whether and the extent to which the due dates are missed. This article shows how Variable Neighborhood Descent and Variable Neighborhood Search can be applied in order to minimize the total tardiness of a given set of customer orders. In a series of extensive numerical experiments, the performance of the two approaches is analyzed for different problem classes. It is shown that the proposed methods provide solutions which may allow order picking systems to operate more efficiently

Automated Order Picking Systems and the Links between Design and Performance: A Systematic Literature Review

With new market developments and e-commerce, there is an increased use of and interest in automation for order picking. This paper presents a systematic review and content analysis of the literature. It has the purpose of understanding the relevant performance aspects for automated, or partly automated, OPSs and identifying the studied links between design and performance, i.e. identifying which combinations of design aspects and performance aspects have been studied in previous research. For this purpose, 74 papers were selected and reviewed. From the review, it is clear that there has been an increased number of papers dealing with the performance of automated, or partly automated, OPSs in recent years. Moreover, there are differences between the different OPS types, but, overall, the performance categories of throughput, lead time, and operational efficiency have received the most attention in the literature. The paper identifies links between design and performance that have been studied, as well as links that appear to be under-researched. For academics, this paper synthesises the current knowledge on the performance of automation in OPSs and identifies opportunities for future research. For practitioners, the paper provides knowledge that can support the decision-making process of automation in OPSs