Deep controlled learning of dynamic policies with an application to lost-sales inventory control

Recent literature established that neural networks can represent good policies across a range of stochastic dynamic models in supply chain and logistics. We propose a new algorithm that incorporates variance reduction techniques, to overcome limitations of algorithms typically employed in literature to learn such neural network policies. For the classical lost sales inventory model, the algorithm learns neural network policies that are vastly superior to those learned using model-free algorithms, while outperforming the best heuristic benchmarks by an order of magnitude. The algorithm is an interesting candidate to apply to other stochastic dynamic problems in supply chain and logistics, because the ideas in its development are generic

Estimating obsolescence risk from demand data - a case study

In this paper obsolescence of service parts is analyzed in a practical environment. Based on the analysis, we propose a method that can be used to estimate the risk of obsolescence of service parts. The method distinguishes groups of service parts. For these groups, the risk of obsolescence is estimated using the behavior of similar groups of service parts in the past. The method uses demand data as main information source, and can therefore be applied without the use of an expert's opinion. We will give numerical values for the risk of obsolescence obtained with the method, and the e®ects of these values on inventory control will be examined

Projected Inventory Level Policies for Lost Sales Inventory Systems: Asymptotic Optimality in Two Regimes

We consider the canonical periodic review lost sales inventory system with positive lead-times and stochastic i.i.d. demand under the average cost criterion. We introduce a new policy that places orders such that the expected inventory level at the time of arrival of an order is at a fixed level and call it the Projected Inventory Level (PIL) policy. We prove that this policy has a cost-rate superior to the equivalent system where excess demand is back-ordered instead of lost and is therefore asymptotically optimal as the cost of losing a sale approaches infinity under mild distributional assumptions. We further show that this policy dominates the constant order policy for any finite lead-time and is therefore asymptotically optimal as the lead-time approaches infinity for the case of exponentially distributed demand per period. Numerical results show this policy also performs superior relative to other policies

Spare parts inventory control for an aircraft component repair shop

We study spare parts inventory control for a repair shop for aircraft components. Defect components that are removed from the aircraft are sent to such a shop for repair. Only after inspection of the component, it becomes clear which specific spare parts are needed to repair it, and in what quantity they are needed. Market requirements on shop performance are reflected in fill rate requirements on the turn around times of the repairs for each component type. The inventory for spare parts is controlled by independent min-max policies. Because parts may be used in the repair of different component types, the resulting optimization problem has a combinatorial nature. Practical instances may consist of 500 component types and 4000 parts, and thus pose a significant computational challenge. We propose a solution algorithm based on column generation. We study the pricing problem, and develop a method that is very efficient in (repeatedly) solving this pricing problem. With this method, it becomes feasible to solve practical instances of the problem in minutes

Scalable policies for the dynamic traveling multi-maintainer problem with alerts

Downtime of industrial assets such as wind turbines and medical imaging devices is costly. To avoid such downtime costs, companies seek to initiate maintenance just before failure, which is challenging because: (i) Asset failures are notoriously difficult to predict, even in the presence of real-time monitoring devices which signal degradation; and (ii) Limited resources are available to serve a network of geographically dispersed assets. In this work, we study the dynamic traveling multi-maintainer problem with alerts (K-DTMPA) under perfect condition information with the objective to devise scalable solution approaches to maintain large networks with K maintenance engineers. Since such large-scale K-DTMPA instances are computationally intractable, we propose an iterative deep reinforcement learning (DRL) algorithm optimizing long-term discounted maintenance costs. The efficiency of the DRL approach is vastly improved by a reformulation of the action space (which relies on the Markov structure of the underlying problem) and by choosing a smart, suitable initial solution. The initial solution is created by extending existing heuristics with a dispatching mechanism. These extensions further serve as compelling benchmarks for tailored instances. We demonstrate through extensive numerical experiments that DRL can solve single maintainer instances up to optimality, regardless of the chosen initial solution. Experiments with hospital networks containing up to 35 assets show that the proposed DRL algorithm is scalable. Lastly, the trained policies are shown to be robust against network modifications such as removing an asset or an engineer or yield a suitable initial solution for the DRL approach.</p

Risk-based stock decisions for projects

In this report we discuss a model that can be used to determine stocking levels using the data that comes forward from a Shell RCM analysis and the data available in E-SPIR. The model is appropriate to determine stock quantities for parts that are used in redundancy situations, and for parts that are used in different pieces of equipment with different downtime costs. Estimating the annual production loss using the model consists of a number of steps. First, we need to determine which spares are used for the repairs of which failure modes. In the second step, we estimate the average waiting time for spares as a function of the number of spares stocked. In the third step, the annual downtime costs are determined. We combine the downtime costs with the holding costs to determine the optimal number of parts to stock

Integrating Reliability Centered Maintenance and Spare Parts Stock Control

In the classical approach to determine how many spare parts to stock, the spare parts shortage costs or the minimum fill rate are a key factor. A difficulty with this approach lies in the estimation of these shortage costs or the determination of appropriate minimum fill rates. In an attempt to overcome this problem, we propose to use the data gathered in reliability centered maintenance studies to determine shortage costs. We discuss benefits of this approach. At the same time, the approach gives rise to complications, as the RCM study determines downtime costs of the underlying equipment, which have a complex relation with the shortage cost for spare parts in case multiple pieces of equipment have different downtime costs. A further complication is redundancy in the equipment. We develop a framework that enables the modelling of these more complicated systems. We propose an approximative, analytic method based on the model that can be used to determine minimum stock quantities in case of redundancy and multiple systems. In a quantitative study we show that the method performs well. Moreover, we show that including redundancy information in the stocking decision gives significant cost benefits

Finding optimal policies in the (S - 1, S ) lost sales inventory model with multiple demand classes

This paper examines the algorithms proposed in the literature for finding good critical level policies in the (S-1,S) lost sales inventory model with multiple demand classes. Our main result is that we establish guaranteed optimality for two of these algorithms. This result is extended to different resupply assumptions, such as a single server queue. As a corollary, we provide an alternative proof of the optimality of critical level policies among the class of all policies

Redesign Policy for a System with Uncertain Failure Rates

Consider a newly developed system sold under a performance based contract (PBC), and subject to failures following various failure modes. Redesign may address certain failure modes. To trade-off redesign costs and costs associated with the PBC, we find the optimal failure modes to redesign based on the current beliefs on the failure rate per failure mode, and the amount of time remaining under the PBC. We develop analytic insights into the structure of the optimal policy

Maintenance Centered Service Parts Inventory Control

High-tech capital goods enable the production of many services and articles that have become a part of our daily lives. Examples include the refineries that produce the gasoline we put in our cars, the photolithography systems that enable the production of the chips in our cell phones and laptops, the trains and railway infrastructure that facilitate public transport and the aircraft that permit us to travel long distances. To prevent costly production disruptions of such systems when failures occur, it is crucial that service parts are readily available to replace any failed parts. However, service parts represent significant investments and failures are unpredictable, so it is unclear which parts should be stocked and in what quantity. In this thesis, analytical models and solution methods are developed to aid companies in making this decision. Amongst other things, we analyze systems in which multiple parts need replacement after a failure, a situation that is frequently encountered in practice. This affects the ability to complete repairs in a timely fashion. We develop new modeling techniques in order to successfully apply scalable deterministic approaches, such as column generation techniques and sample average approximation methods, to this stochastic problem. This leads to solution techniques that, unlike traditional methods, can ensure that all parts needed to complete maintenance are readily available. The approach is capable of meeting the challenging requirements of a real-life repair shop