Reinforcement Learning Approaches for the Orienteering Problem with Stochastic and Dynamic Release Dates

In this paper, we study a sequential decision making problem faced by e-commerce carriers related to when to send out a vehicle from the central depot to serve customer requests, and in which order to provide the service, under the assumption that the time at which parcels arrive at the depot is stochastic and dynamic. The objective is to maximize the number of parcels that can be delivered during the service hours. We propose two reinforcement learning approaches for solving this problem, one based on a policy function approximation (PFA) and the second on a value function approximation (VFA). Both methods are combined with a look-ahead strategy, in which future release dates are sampled in a Monte-Carlo fashion and a tailored batch approach is used to approximate the value of future states. Our PFA and VFA make a good use of branch-and-cut-based exact methods to improve the quality of decisions. We also establish sufficient conditions for partial characterization of optimal policy and integrate them into PFA/VFA. In an empirical study based on 720 benchmark instances, we conduct a competitive analysis using upper bounds with perfect information and we show that PFA and VFA greatly outperform two alternative myopic approaches. Overall, PFA provides best solutions, while VFA (which benefits from a two-stage stochastic optimization model) achieves a better tradeoff between solution quality and computing time

A survey of scheduling problems with setup times or costs

Author name used in this publication: C. T. NgAuthor name used in this publication: T. C. E. Cheng2007-2008 > Academic research: refereed > Publication in refereed journalAccepted ManuscriptPublishe

Design and Control of Warehouse Order Picking: a literature review

Order picking has long been identified as the most labour-intensive and costly activity for almost every warehouse; the cost of order picking is estimated to be as much as 55% of the total warehouse operating expense. Any underperformance in order picking can lead to unsatisfactory service and high operational cost for its warehouse, and consequently for the whole supply chain. In order to operate efficiently, the orderpicking process needs to be robustly designed and optimally controlled. This paper gives a literature overview on typical decision problems in design and control of manual order-picking processes. We focus on optimal (internal) layout design, storage assignment methods, routing methods, order batching and zoning. The research in this area has grown rapidly recently. Still, combinations of the above areas have hardly been explored. Order-picking system developments in practice lead to promising new research directions.Order picking;Logistics;Warehouse Management

Mathematical models for planning support

In this paper we describe how computer systems can provide planners with active planning support, when these planners are carrying out their daily planning activities. This means that computer systems actively participate in the planning process by automatically generating plans or partial plans. Active planning support by computer systems requires the application of mathematical models and solution techniques. In this paper we describe the modeling process in general terms, as well as several modeling and solution techniques. We also present some background information on computational complexity theory, since most practical planning problems are hard to solve. We also describe how several objective functions can be handled, since it is rare that solutions can be evaluated by just one single objective. Furthermore, we give an introduction into the use of mathematical modeling systems, which are useful tools in a modeling context, especially during the development phases of a mathematical model. We finish the paper with a real life example related to the planning process of the rolling stock circulation of a railway operator.optimization;mathematical models;modeling process;planning support;Planning

Strategies for Handling Temporal Uncertainty in Pickup and Delivery Problems with Time Windows

In many real-life routing problems there is more uncertainty with respect to the required timing of the service than with respect to the service locations. We focus on a pickup and delivery problem with time windows in which the pickup and drop-off locations of the service requests are fully known in advance, but the time at which these jobs will require service is only fully revealed during operations. We develop a sample-scenario routing strategy to accommodate a variety of potential time real- izations while designing and updating the routes. Our experiments on a breadth of instances show that advance time related information, if used intelligently, can yield benefits. Furthermore, we show that it is beneficial to tailor the consensus function that is used in the sample-scenario approach to the specifics of the problem setting. By doing so, our strategy performs well on instances with both short time windows and limited advance confirmation

The Dynamic Multi-objective Multi-vehicle Covering Tour Problem

This work introduces a new routing problem called the Dynamic Multi-Objective Multi-vehicle Covering Tour Problem (DMOMCTP). The DMOMCTPs is a combinatorial optimization problem that represents the problem of routing multiple vehicles to survey an area in which unpredictable target nodes may appear during execution. The formulation includes multiple objectives that include minimizing the cost of the combined tour cost, minimizing the longest tour cost, minimizing the distance to nodes to be covered and maximizing the distance to hazardous nodes. This study adapts several existing algorithms to the problem with several operator and solution encoding variations. The efficacy of this set of solvers is measured against six problem instances created from existing Traveling Salesman Problem instances which represent several real countries. The results indicate that repair operators, variable length solution encodings and variable-length operators obtain a better approximation of the true Pareto front