A Sampling-Based Method for Gittins Index Approximation

A sampling-based method is introduced to approximate the Gittins index for a general family of alternative bandit processes. The approximation consists of a truncation of the optimization horizon and support for the immediate rewards, an optimal stopping value approximation, and a stochastic approximation procedure. Finite-time error bounds are given for the three approximations, leading to a procedure to construct a confidence interval for the Gittins index using a finite number of Monte Carlo samples, as well as an epsilon-optimal policy for the Bayesian multi-armed bandit. Proofs are given for almost sure convergence and convergence in distribution for the sampling based Gittins index approximation. In a numerical study, the approximation quality of the proposed method is verified for the Bernoulli bandit and Gaussian bandit with known variance, and the method is shown to significantly outperform Thompson sampling and the Bayesian Upper Confidence Bound algorithms for a novel random effects multi-armed bandit

Initial spare parts assortment decision making for rolling stock maintenance: a structured approach

Design for maintenance and maintenance operations become increasingly important in recent years. In the capital-intensive industry, maintenance expenditures can add up to several times the initial investment. In order to be competitive in their business, owners and users of these capital goods have to take into account the total life cycle cost at investment (the lifespan of a train is about 30 years), the renewal decisions for their installations and the logistic management of the spare parts. Erroneous or unstructured initial spare parts assortment decision-making part of the logistic management can lead to undesired downtime and increases the risk of obsolete or unavailable components. Decision making is further complicated by non- existent data in the early design phase and several information management problems. Based on a case study at NedTrain (the largest maintainer of rolling stock in the Netherlands) and literature review a Decision Support Model to structure and to improve the initial spare part assortment for the rolling stock maintenance is proposed